-- Logs begin at Thu 2022-08-04 01:10:24 UTC, end at Thu 2022-08-04 02:37:18 UTC. -- Aug 04 02:03:13.169846 dv-cinder-740146-17-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 04 02:03:13.179498 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 04 02:03:13.181597 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Aug 4 02:03:13 2022] *** Aug 04 02:03:13.181597 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 04 02:03:13.181597 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 04 02:03:13.181597 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: nodename: dv-cinder-740146-17-iscsi Aug 04 02:03:13.181597 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: machine: x86_64 Aug 04 02:03:13.181597 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: clock source: unix Aug 04 02:03:13.181597 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: pcre jit disabled Aug 04 02:03:13.181597 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: detected number of CPU cores: 4 Aug 04 02:03:13.181597 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: current working directory: / Aug 04 02:03:13.181597 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: detected binary path: /usr/bin/uwsgi-core Aug 04 02:03:13.181597 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: your processes number limit is 47680 Aug 04 02:03:13.181597 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: your memory page size is 4096 bytes Aug 04 02:03:13.181597 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: detected max file descriptor number: 2048 Aug 04 02:03:13.181597 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: lock engine: pthread robust mutexes Aug 04 02:03:13.182380 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: thunder lock: enabled Aug 04 02:03:13.182380 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 04 02:03:13.182380 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 04 02:03:13.224825 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: Python main interpreter initialized at 0x56377f4d99c0 Aug 04 02:03:13.224825 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: python threads support enabled Aug 04 02:03:13.224825 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: your server socket listen backlog is limited to 100 connections Aug 04 02:03:13.224825 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: your mercy for graceful operations on workers is 90 seconds Aug 04 02:03:13.225228 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: mapped 403077 bytes (393 KB) for 2 cores Aug 04 02:03:13.225228 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: *** Operational MODE: preforking *** Aug 04 02:03:13.225506 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: *** uWSGI is running in multiple interpreter mode *** Aug 04 02:03:13.225506 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: spawned uWSGI master process (pid: 80660) Aug 04 02:03:13.225718 dv-cinder-740146-17-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 04 02:03:13.227247 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: spawned uWSGI worker 1 (pid: 80672, cores: 1) Aug 04 02:03:13.227247 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: spawned uWSGI worker 2 (pid: 80673, cores: 1) Aug 04 02:03:13.227247 dv-cinder-740146-17-iscsi devstack@keystone.service[80660]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 04 02:03:18.773115 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 02:03:18.773115 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: from cryptography.utils import int_from_bytes Aug 04 02:03:18.773912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 02:03:18.773912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: from cryptography.utils import int_from_bytes Aug 04 02:03:18.775685 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 02:03:18.775685 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: from cryptography.utils import int_from_bytes Aug 04 02:03:18.776144 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 04 02:03:18.776144 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: from cryptography.utils import int_from_bytes Aug 04 02:03:18.823742 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 02:03:18.823926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 02:03:18.824120 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] command line args: [] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 02:03:18.824390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 02:03:18.824567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ================================================================================ {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 02:03:18.824851 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] admin_token = **** {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.825100 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] config_dir = [] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.825289 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.825365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 04 02:03:18.825472 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] config_source = [] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.825550 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 04 02:03:18.825651 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] debug = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.825734 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] command line args: [] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 04 02:03:18.825877 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.826108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 04 02:03:18.826108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ================================================================================ {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 04 02:03:18.826235 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.826356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] admin_token = **** {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.826436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.826595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] config_dir = [] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.826679 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.826780 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.826860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.826966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] config_source = [] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.827070 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] list_limit = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.827147 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] debug = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.827243 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] log_config_append = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.827340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.827475 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.827577 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.827676 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] log_dir = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.827792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.827886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] log_file = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.827985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.828087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.828185 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.828260 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.828417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] list_limit = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.828504 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.828656 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] log_config_append = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.828732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.828803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.828916 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.829001 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] log_dir = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.829078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.829203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] log_file = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.829283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.829358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.829506 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.829585 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.829741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.829821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.829973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.830095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.830180 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.830256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.830376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.830461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.830579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.830763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.831367 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.831451 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.831604 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.831690 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.831837 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.832070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.832186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.832302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.832381 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] publish_errors = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.832548 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.832647 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.832827 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.832915 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.832993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.833103 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.833240 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.833478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.833600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.833715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] standard_threads = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.833799 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] publish_errors = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.833948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.834042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.834193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.834269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.834371 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.834447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.834568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] use_journal = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.834641 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.834710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] use_json = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.834847 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.834922 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] use_stderr = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.835037 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] use_syslog = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.835167 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] standard_threads = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.835272 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.835354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.835531 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.835608 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.835706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.835792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.835884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] use_journal = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.836023 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.836105 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] use_json = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.836224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] use_stderr = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.836305 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.836400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] use_syslog = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.836503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.836662 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 04 02:03:18.836769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.836920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.837006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.837162 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.837239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.837423 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.837499 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.837665 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.837747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.837915 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.837994 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.838154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.838237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.838403 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.838479 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.838642 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.838723 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.838890 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.838966 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.839137 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.839212 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.839375 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.839456 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.839611 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.839685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.839860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.839939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.840084 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.840163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.840328 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.840410 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.840576 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.840669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.840837 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.840916 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.841090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.841178 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.841346 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.841421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.841580 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.841661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.841818 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.841894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.842053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.842134 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.842294 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.842377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.842546 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.842622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.842795 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.842881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.843028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.843104 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.843258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.843338 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.843490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.843575 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.843727 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.843804 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.843971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.844052 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.844220 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.844303 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.844453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.844528 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.844704 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.844793 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.844950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.845034 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.845186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.845262 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.845420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.845495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.845656 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.845810 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.845883 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.845956 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.846115 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.846196 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.846457 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.846544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.846770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.846850 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.847072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.847310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.847552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.847757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.847843 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.848007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.848122 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.848250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.848352 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.848493 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.848595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.848749 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.848861 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.848991 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.849131 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.849223 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.849295 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.849462 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.849540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.849695 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.849803 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.849938 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.850058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.850179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.850261 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.850415 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.850493 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.850653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.850732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.850892 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.850971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.851126 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.851212 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.851361 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.851449 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.851606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.851691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.851847 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.851925 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.852082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.852167 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.852324 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.852407 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.852564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.852711 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.852820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.852930 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.853061 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.853179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.853300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.853431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.853569 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.853680 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.853810 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.853919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.854061 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.854158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.854313 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.854397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.854550 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.854627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.854787 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.854863 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.855023 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.855107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.855273 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.855349 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.855514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.855596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.855754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.855843 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.855996 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.856071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.856232 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.856304 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.856469 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.856553 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.856726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.856805 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.856968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.857044 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.857206 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.857294 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.857462 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.857538 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.857706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.857787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.857943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.858096 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.858172 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.858239 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.858405 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.858482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.858638 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.858723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.858871 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.858967 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.859104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.859180 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.859334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.859409 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.859564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.859647 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.859796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.859871 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.860031 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.860107 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.860262 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.860344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.860504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.860581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.860765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.860843 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.861015 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.861103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.861255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.861332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.861490 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] auth.external = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.861570 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.861726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.861810 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.862017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.862096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.862254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] auth.external = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.862331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.862489 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.862576 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] auth.password = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.862725 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] auth.token = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.862803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.862969 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.863045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.863209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.863294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] auth.password = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.863450 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] auth.token = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.863527 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.863685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.863760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.863923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.863999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.864173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.864251 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.864405 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.864480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.864676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.864792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] credential.caching = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.864930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.865015 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.865167 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.865243 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.865406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] credential.caching = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.865483 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.865650 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.865727 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.865900 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.865976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.866138 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.866215 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.866429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.866525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.866629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.866708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.866867 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.866943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.867124 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.867317 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.867392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.867461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.867633 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.867720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.867850 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.867929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.868131 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.868215 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.868379 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.868455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.868661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] federation.caching = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.868744 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.868880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.868958 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.869130 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.869207 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.869379 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.869458 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] federation.caching = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.869613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.869691 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.869861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.869936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.870101 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.870182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.870333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.870409 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.870585 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.870662 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.870826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.870904 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.871065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.871141 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.871311 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.871387 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.871548 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] identity.caching = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.871625 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.871794 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.871869 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.872043 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.872120 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.872266 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.872339 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] identity.caching = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.872507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.872585 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.872777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.872865 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.873011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.873088 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.873250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.873326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.873497 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.873580 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.873731 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.873807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.873969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.874043 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.874200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.874287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.874433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.874510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.874681 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.874758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.874912 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.874996 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.875147 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.875224 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.875394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.875473 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.875642 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.875727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.875891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.875970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.876143 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.876218 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.876384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.876469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.876650 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.876732 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.876878 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.876955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.877114 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.877203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.877355 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.877428 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.878108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.878191 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.878346 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.878437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.878586 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.878664 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.878820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.878896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.879059 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.879143 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.879311 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.879387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.879544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.879620 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.879791 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.879875 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.880039 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.880115 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.880274 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.880349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.880515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.880600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.880780 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.880859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.881014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.881089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.881252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.881339 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.881484 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.881559 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.881723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.881800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.881959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.882045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.882202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.882279 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.882441 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.882519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.882680 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.882766 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.882914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.883000 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.883164 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.883241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.883417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.883503 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.883649 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.883725 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.883886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.883962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.884124 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.884210 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.884358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.884435 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.884599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.884722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.884864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.884942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.885098 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.885175 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.885341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.885426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.885590 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.885667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.885824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.885901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.886063 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.886139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.886296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.886375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.886534 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.886611 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.886772 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.886849 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.887002 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.887080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.887233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.887310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.887468 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.887543 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.887705 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.887790 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.887938 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.888015 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.888187 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.888306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.888435 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.888557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.888703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.888819 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.888954 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.889073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.889203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.889314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.889438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.889556 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.889691 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.889807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.889930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.890033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.890167 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.890282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.890404 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.890534 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.890769 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.890848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.891019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.891097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.891250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.891333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.891496 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.891575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.891734 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.891810 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.891971 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.892047 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.892207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.892285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.892443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.892519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.892701 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.892784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.892941 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.893020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.893256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.893848 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.894015 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.894092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] resource.caching = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.894258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.894338 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.894501 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.894579 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.894741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.894817 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] resource.caching = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.894996 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.895075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.895247 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.895324 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.895485 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.895560 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.895737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.895814 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.895977 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.896052 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.896220 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.896295 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.896458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] role.caching = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.896537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.896722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.896803 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] role.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.896958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.897034 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.897196 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] role.caching = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.897273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.897446 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] role.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.897525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.897681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.897758 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.897932 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.898011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.898179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.898255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.898997 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.899110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.899186 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.899258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.899317 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.899376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.899376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.899510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.899582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.899651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.899719 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.899788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.899857 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.899932 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.900081 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.900160 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.900331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.900523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.900600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.900798 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.900880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.901052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.901261 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.901340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.901500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.901587 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.901752 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.901830 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.902003 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.902084 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.902254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.902339 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.902507 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.902585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.902744 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.902822 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.902978 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.903064 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.903214 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.903292 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.903451 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.903528 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.903695 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.903773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.903930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.904008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.904167 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.904244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.904421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.904499 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.904684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.904767 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.904921 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.904998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.905161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] token.caching = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.905241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.905400 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.905477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.905632 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.905708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.905871 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] token.caching = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.905950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.906109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.906186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.906356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.906434 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.906595 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.906672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.906841 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.906919 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.907132 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.907216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.907325 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.907404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.907565 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.907641 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.907806 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.907883 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.908043 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.908122 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.908279 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.908361 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.908521 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.908598 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.908804 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.908892 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.909053 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.909131 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.909303 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.909378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.909536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.909647 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.connection = **** {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.909780 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.909895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.910074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.910152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.910336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.connection = **** {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.910426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.910582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.910669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.910826 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.910903 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.911066 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.911143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.911302 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.911387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.911537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.911614 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.911775 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.911852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.912011 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.912097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.912249 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.912327 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.912485 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.912560 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.912742 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.912834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.913036 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.913130 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.913225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.913303 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.913458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.913543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.913699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.913776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.913936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.914013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.914109 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80673) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 02:03:18.914196 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.914397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.914633 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 04 02:03:18.914807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80672) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 04 02:03:18.943880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:18.944107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:18.944336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:18.944419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:18.944679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:18.944911 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:18.952726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:18.953262 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:18.982881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:18.983112 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:18.983327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:18.983561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.035730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.036054 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.086681 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.086908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.087004 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.087148 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.151524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.151748 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.151914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.152003 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.152151 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.152236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.152378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.152459 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.152592 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.152701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.152835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.153051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.176204 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.176391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.176600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.176732 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.176865 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.176950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.177085 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.177167 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.177301 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.177376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.177514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.177593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.177727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.177941 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.188848 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.189153 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.202320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.202552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.202641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.202779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.202863 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.203005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.203087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.203226 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.203317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.203439 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.203519 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.203651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80672) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.203732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.203861 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80673) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 04 02:03:19.220899 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.221112 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 02:03:19.221205 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.221402 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 02:03:19.221496 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.221660 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 02:03:19.221743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.221928 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 04 02:03:19.222007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.222186 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.222273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.222442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.222522 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.222691 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.222768 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.222943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.232409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.232718 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.232804 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.232987 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.233069 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.233265 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.233348 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.233547 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.240685 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.240990 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.245446 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.245726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.245810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.246006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.246086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.246281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.246370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.246558 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.246645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.246818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.246900 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.247077 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.247158 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.247345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.247424 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.247609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.247688 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.247874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.247952 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.248133 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.248211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.248393 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.271532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.271782 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.273044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.273303 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.273386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.273582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.277929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.278208 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.278292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.278487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.278567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.278766 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.285268 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.285406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.285568 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.285690 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.289684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.289797 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.289973 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.290080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.290254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.290354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.290556 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.290675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.290842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.290934 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.291135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.291226 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.291405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.291493 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.291671 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.291764 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.308054 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.308160 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.308324 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.308406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.312534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.312680 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.316292 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.316392 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.316582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.316692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.321883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.322030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.325058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.325461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.328222 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.328514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.328656 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.328824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.328957 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.329119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.329251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.329446 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.329537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.329706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.329786 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.330064 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.345695 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.345974 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.346186 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.346806 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.346981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.347071 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.347246 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.347513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.353858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.354361 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.355418 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.355692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.355928 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.356203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.358552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.358831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.359049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.359130 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.359331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.359411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.359619 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.359707 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.359895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.360182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.371624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.371912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.372153 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.372236 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.372432 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.372514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.372742 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.372843 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.373013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.373090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.373290 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.373377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.373570 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.373850 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.388852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.389443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.391822 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.392410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.394826 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.395400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.397407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.397991 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.398916 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.399201 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.399488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.399568 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.399771 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.400052 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.405921 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.406507 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.407539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.408113 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.411019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.411305 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.411570 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.411653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.411867 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.411959 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.412132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.412406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.419887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.420179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.420429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.420511 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.420744 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.421021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.427934 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80673) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.428161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.428426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80672) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 04 02:03:19.428518 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.428734 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.428817 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.429004 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.429085 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.429271 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.429364 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.429538 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.429620 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.429805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.429883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.430078 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.430191 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.430361 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.430449 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.430721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.430802 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.430990 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.431071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.431259 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.431346 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.431517 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80673) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.431598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.431780 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.432048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80672) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 04 02:03:19.464960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56377f4d99c0 pid: 80673 (default app) Aug 04 02:03:19.468329 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463fafa-3289-498d-ab49-3bb3c88212b5 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:19.468638 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463fafa-3289-498d-ab49-3bb3c88212b5 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:19.468927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463fafa-3289-498d-ab49-3bb3c88212b5 None None] PATH_INFO: `/v3` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:19.471024 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56377f4d99c0 pid: 80672 (default app) Aug 04 02:03:19.473824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1/1] 10.222.0.45 () {50 vars in 791 bytes} [Thu Aug 4 02:03:19 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:27.225325 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe8440-d7fb-46cd-985a-3074deb1c533 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:27.225885 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe8440-d7fb-46cd-985a-3074deb1c533 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:27.225885 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe8440-d7fb-46cd-985a-3074deb1c533 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:27.230303 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1/2] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:03:27.235704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0315477-fca1-457c-b12f-bdbdf83f9ef6 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:27.236133 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0315477-fca1-457c-b12f-bdbdf83f9ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:27.236254 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0315477-fca1-457c-b12f-bdbdf83f9ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:27.308089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: INFO dbcounter [None req-c0315477-fca1-457c-b12f-bdbdf83f9ef6 None None] Registered counter for database keystone Aug 04 02:03:27.327808 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG oslo_db.sqlalchemy.engines [None req-c0315477-fca1-457c-b12f-bdbdf83f9ef6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80673) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 02:03:27.334064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writer thread running {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 02:03:27.516307 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG passlib.handlers.bcrypt [None req-c0315477-fca1-457c-b12f-bdbdf83f9ef6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80673) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 02:03:27.516645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG passlib.handlers.bcrypt [None req-c0315477-fca1-457c-b12f-bdbdf83f9ef6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80673) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 02:03:27.576174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-c0315477-fca1-457c-b12f-bdbdf83f9ef6 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:27.577846 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-c0315477-fca1-457c-b12f-bdbdf83f9ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:27.708567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 2/3] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 04 02:03:27.864560 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe8440-d7fb-46cd-985a-3074deb1c533 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:27.934178 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: INFO dbcounter [None req-09fe8440-d7fb-46cd-985a-3074deb1c533 None None] Registered counter for database keystone Aug 04 02:03:27.955689 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG oslo_db.sqlalchemy.engines [None req-09fe8440-d7fb-46cd-985a-3074deb1c533 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80672) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 04 02:03:27.962435 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writer thread running {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 04 02:03:28.088608 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe8440-d7fb-46cd-985a-3074deb1c533 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:28.093441 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d89c70f-f44e-4f94-8bbc-e2260fdd43e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:28.094073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89c70f-f44e-4f94-8bbc-e2260fdd43e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:28.094352 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89c70f-f44e-4f94-8bbc-e2260fdd43e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:28.094625 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d89c70f-f44e-4f94-8bbc-e2260fdd43e1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:28.103554 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89c70f-f44e-4f94-8bbc-e2260fdd43e1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:28.154943 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5d89c70f-f44e-4f94-8bbc-e2260fdd43e1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 02:03:28.155283 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d89c70f-f44e-4f94-8bbc-e2260fdd43e1 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:28.165794 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 2/4] 10.222.0.45 () {58 vars in 1324 bytes} [Thu Aug 4 02:03:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 02:03:28.171767 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0315477-fca1-457c-b12f-bdbdf83f9ef6 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:28.186420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0315477-fca1-457c-b12f-bdbdf83f9ef6 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:28.190790 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebcc29b-1787-403d-b5f8-74207d20866c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:28.191904 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcc29b-1787-403d-b5f8-74207d20866c admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:28.192190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcc29b-1787-403d-b5f8-74207d20866c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:28.192467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebcc29b-1787-403d-b5f8-74207d20866c admin admin] PATH_INFO: `/v3/projects/457cd62a685649778d5130f3937b1bcf` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:28.194022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebcc29b-1787-403d-b5f8-74207d20866c admin admin] RBAC: Authorizing `identity:get_project(project_id=457cd62a685649778d5130f3937b1bcf)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:28.244274 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: WARNING keystone.common.rbac_enforcer.enforcer [None req-eebcc29b-1787-403d-b5f8-74207d20866c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 04 02:03:28.244878 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebcc29b-1787-403d-b5f8-74207d20866c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:28.249330 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 3/5] 10.222.0.45 () {56 vars in 1207 bytes} [Thu Aug 4 02:03:28 2022] GET /identity/v3/projects/457cd62a685649778d5130f3937b1bcf => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:30.210373 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73168e-ac65-444d-803c-7eec907bfb93 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:30.210892 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73168e-ac65-444d-803c-7eec907bfb93 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:30.210996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e73168e-ac65-444d-803c-7eec907bfb93 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:30.215073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 3/6] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:30.220520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaefd2-3307-4031-bdd0-690cc98743b0 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:30.220979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaefd2-3307-4031-bdd0-690cc98743b0 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:30.221102 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaefd2-3307-4031-bdd0-690cc98743b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:30.241368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-bbfaefd2-3307-4031-bdd0-690cc98743b0 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:30.242884 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-bbfaefd2-3307-4031-bdd0-690cc98743b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:30.250964 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 4/7] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 04 02:03:30.273237 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73168e-ac65-444d-803c-7eec907bfb93 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:30.285790 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e73168e-ac65-444d-803c-7eec907bfb93 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:30.289937 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df3ece-a683-44fa-a15b-184e55c30d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:30.290535 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df3ece-a683-44fa-a15b-184e55c30d51 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:30.290846 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df3ece-a683-44fa-a15b-184e55c30d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:30.291096 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df3ece-a683-44fa-a15b-184e55c30d51 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:30.298689 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df3ece-a683-44fa-a15b-184e55c30d51 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:30.300483 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df3ece-a683-44fa-a15b-184e55c30d51 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:30.309231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 4/8] 10.222.0.45 () {58 vars in 1324 bytes} [Thu Aug 4 02:03:30 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 02:03:30.314899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfaefd2-3307-4031-bdd0-690cc98743b0 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:30.323894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfaefd2-3307-4031-bdd0-690cc98743b0 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:30.328221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7ed4fb-2df3-4eb3-82b1-81363f9318a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:30.328883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ed4fb-2df3-4eb3-82b1-81363f9318a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:30.329169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ed4fb-2df3-4eb3-82b1-81363f9318a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:30.329446 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7ed4fb-2df3-4eb3-82b1-81363f9318a4 admin admin] PATH_INFO: `/v3/users/1cdae040f69d4994bc484e82a80c2139` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:30.330797 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ed4fb-2df3-4eb3-82b1-81363f9318a4 admin admin] RBAC: Authorizing `identity:get_user(user_id=1cdae040f69d4994bc484e82a80c2139)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:30.333183 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7ed4fb-2df3-4eb3-82b1-81363f9318a4 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:30.334718 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 5/9] 10.222.0.45 () {56 vars in 1198 bytes} [Thu Aug 4 02:03:30 2022] GET /identity/v3/users/1cdae040f69d4994bc484e82a80c2139 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:32.402963 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4898d0-33b0-46dc-a584-b6daaf36b3fb None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:32.403505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4898d0-33b0-46dc-a584-b6daaf36b3fb None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:32.403505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4898d0-33b0-46dc-a584-b6daaf36b3fb None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:32.403714 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c708b14-75b7-4609-a49a-ac7ca438f14f None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:32.404138 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c708b14-75b7-4609-a49a-ac7ca438f14f None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:32.404276 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c708b14-75b7-4609-a49a-ac7ca438f14f None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:32.404855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 5/10] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:32.405715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 6/11] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:03:32.409962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b82bb3-5213-4d30-b01e-ef810d0353c5 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:32.410244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b82bb3-5213-4d30-b01e-ef810d0353c5 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:32.410516 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b82bb3-5213-4d30-b01e-ef810d0353c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:32.410969 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e360ff-fd86-4516-8095-497d6357a679 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:32.411259 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e360ff-fd86-4516-8095-497d6357a679 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:32.411542 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e360ff-fd86-4516-8095-497d6357a679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:32.430422 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-e6e360ff-fd86-4516-8095-497d6357a679 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:32.431618 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e6e360ff-fd86-4516-8095-497d6357a679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:32.453085 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG passlib.handlers.bcrypt [None req-b5b82bb3-5213-4d30-b01e-ef810d0353c5 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80672) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 04 02:03:32.453504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG passlib.handlers.bcrypt [None req-b5b82bb3-5213-4d30-b01e-ef810d0353c5 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80672) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 04 02:03:32.501366 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-b5b82bb3-5213-4d30-b01e-ef810d0353c5 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:32.502977 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b5b82bb3-5213-4d30-b01e-ef810d0353c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:32.514218 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 7/12] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:03:32.536012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e360ff-fd86-4516-8095-497d6357a679 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:32.559799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:32.604702 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 6/13] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:03:32.627614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b82bb3-5213-4d30-b01e-ef810d0353c5 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:32.639017 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e360ff-fd86-4516-8095-497d6357a679 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:32.672360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f5efd-1f11-4f1a-a912-a3c3e49ddf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:32.673038 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f5efd-1f11-4f1a-a912-a3c3e49ddf7a None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:32.673327 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f5efd-1f11-4f1a-a912-a3c3e49ddf7a None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:32.673607 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f5efd-1f11-4f1a-a912-a3c3e49ddf7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:32.734653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b82bb3-5213-4d30-b01e-ef810d0353c5 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:32.736970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:32.752544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f5efd-1f11-4f1a-a912-a3c3e49ddf7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:32.754682 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f5efd-1f11-4f1a-a912-a3c3e49ddf7a None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:32.777263 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8d9eb-059a-43b8-85e6-4432c8229fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:32.778530 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8d9eb-059a-43b8-85e6-4432c8229fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:32.778814 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8d9eb-059a-43b8-85e6-4432c8229fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:32.779038 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:32.779171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8d9eb-059a-43b8-85e6-4432c8229fb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:32.790926 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8d9eb-059a-43b8-85e6-4432c8229fb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:03:32.799416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-a8f8d9eb-059a-43b8-85e6-4432c8229fb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:03:32.799416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:32.799416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:32.799416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:32.799416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:32.799416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:32.799416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:32.799416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:32.799416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:32.799416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:32.799416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:32.799416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:32.799416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:03:32.799416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:03:32.799416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:03:32.799416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:03:32.800908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:03:32.800908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:03:32.800908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:03:32.800908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:03:32.800908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:32.800908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:32.800908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:32.800908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:32.800908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:32.800908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:32.800908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:32.800908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:32.800908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:32.800908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:32.800908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:32.800908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:32.801738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:32.801738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:32.801738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:32.801738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:32.801738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:03:32.801738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:03:32.801738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:03:32.801738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:03:32.801738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:03:32.801738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:03:32.801738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:03:32.801738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:03:32.801738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 7/14] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:03:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:03:32.807049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8d9eb-059a-43b8-85e6-4432c8229fb8 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:32.858581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 8/15] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:03:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 02:03:32.865011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f5efd-1f11-4f1a-a912-a3c3e49ddf7a None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:32.907548 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8d9eb-059a-43b8-85e6-4432c8229fb8 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:32.942827 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b67e2-bcf3-46f0-b737-a05fb2205582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:32.943456 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b67e2-bcf3-46f0-b737-a05fb2205582 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:32.943736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b67e2-bcf3-46f0-b737-a05fb2205582 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:32.944009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171b67e2-bcf3-46f0-b737-a05fb2205582 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:32.944779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b67e2-bcf3-46f0-b737-a05fb2205582 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:32.945071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b67e2-bcf3-46f0-b737-a05fb2205582 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:32.946520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171b67e2-bcf3-46f0-b737-a05fb2205582 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:32.958920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 8/16] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:03:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:03:32.966449 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f5efd-1f11-4f1a-a912-a3c3e49ddf7a None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:32.968172 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b67e2-bcf3-46f0-b737-a05fb2205582 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:32.991028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:32.996849 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:33.004716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0aee14-c581-49bc-afc2-bb3997527116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:33.005578 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0aee14-c581-49bc-afc2-bb3997527116 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:33.005860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0aee14-c581-49bc-afc2-bb3997527116 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:33.006135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0aee14-c581-49bc-afc2-bb3997527116 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:33.016202 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0aee14-c581-49bc-afc2-bb3997527116 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:33.017725 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0aee14-c581-49bc-afc2-bb3997527116 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:33.035770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-8f0aee14-c581-49bc-afc2-bb3997527116 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:03:33.035770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:33.035770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:33.035770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:33.035770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:33.035770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:33.035770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:33.035770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:33.035770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:33.035770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:33.035770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:33.035770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:33.035770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:03:33.035770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:03:33.035770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:03:33.036771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:03:33.036771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:33.036771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:33.036771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:33.036771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:33.036771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:33.036771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:33.036771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:33.036771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:33.036771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:33.036771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:33.036771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:33.036771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:33.036771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:33.036771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:33.036771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:33.036771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:33.037580 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:03:33.037580 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:03:33.037580 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:03:33.037580 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:03:33.038000 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 9/17] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:03:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:03:33.042708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0aee14-c581-49bc-afc2-bb3997527116 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:33.072151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171b67e2-bcf3-46f0-b737-a05fb2205582 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:33.106491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566c628-1331-4367-a401-3ef023d2fcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:33.107156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566c628-1331-4367-a401-3ef023d2fcbf None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:33.107439 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566c628-1331-4367-a401-3ef023d2fcbf None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:33.107716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566c628-1331-4367-a401-3ef023d2fcbf None admin] PATH_INFO: `/v3/users/1cdae040f69d4994bc484e82a80c2139` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:33.109283 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566c628-1331-4367-a401-3ef023d2fcbf None admin] RBAC: Authorizing `identity:get_user(user_id=1cdae040f69d4994bc484e82a80c2139)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:33.112254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566c628-1331-4367-a401-3ef023d2fcbf None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:33.114950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 9/18] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:03:32 2022] GET /identity/v3/users/1cdae040f69d4994bc484e82a80c2139 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:03:33.120508 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566c628-1331-4367-a401-3ef023d2fcbf None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:33.220484 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:33.228249 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566c628-1331-4367-a401-3ef023d2fcbf None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:33.252931 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0aee14-c581-49bc-afc2-bb3997527116 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:33.261606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b8cf7-6c3f-4d55-a076-46f43aa4d25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:33.262254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b8cf7-6c3f-4d55-a076-46f43aa4d25e None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:33.262541 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b8cf7-6c3f-4d55-a076-46f43aa4d25e None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:33.262820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b8cf7-6c3f-4d55-a076-46f43aa4d25e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:33.263982 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b8cf7-6c3f-4d55-a076-46f43aa4d25e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:33.265467 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816b8cf7-6c3f-4d55-a076-46f43aa4d25e None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:33.267108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 10/19] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:03:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:03:33.273360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b8cf7-6c3f-4d55-a076-46f43aa4d25e None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:33.287484 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d1f30-4a62-4da4-aaef-68e648632260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:33.288104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d1f30-4a62-4da4-aaef-68e648632260 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:33.288386 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d1f30-4a62-4da4-aaef-68e648632260 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:33.288684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d1f30-4a62-4da4-aaef-68e648632260 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:33.289381 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d1f30-4a62-4da4-aaef-68e648632260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:33.289685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d1f30-4a62-4da4-aaef-68e648632260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:33.291073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d1f30-4a62-4da4-aaef-68e648632260 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:33.320655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 10/20] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:03:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:33.377770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816b8cf7-6c3f-4d55-a076-46f43aa4d25e None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:33.419790 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d190866b-8a87-47b4-996b-d624f4cb8c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:33.420438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190866b-8a87-47b4-996b-d624f4cb8c1c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:33.420720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190866b-8a87-47b4-996b-d624f4cb8c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:33.421047 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190866b-8a87-47b4-996b-d624f4cb8c1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:33.421873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190866b-8a87-47b4-996b-d624f4cb8c1c None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.domain.id=default, user.id=1cdae040f69d4994bc484e82a80c2139) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:33.422182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190866b-8a87-47b4-996b-d624f4cb8c1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:33.423681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d190866b-8a87-47b4-996b-d624f4cb8c1c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:33.434264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 11/21] 10.222.0.45 () {56 vars in 1322 bytes} [Thu Aug 4 02:03:33 2022] GET /identity/v3/role_assignments?user.id=1cdae040f69d4994bc484e82a80c2139&scope.domain.id=default&role.id=35205394c8af4c52a61b12f92ba16644 => generated 233 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:03:35.685930 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174b293-388e-4c53-a0f9-ce5061e5374f None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:35.686458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174b293-388e-4c53-a0f9-ce5061e5374f None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:35.686458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174b293-388e-4c53-a0f9-ce5061e5374f None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:35.687812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 11/22] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:35.706181 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a969b-2145-419d-b510-f0de54a2157a None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:35.707524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a969b-2145-419d-b510-f0de54a2157a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:35.707524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a969b-2145-419d-b510-f0de54a2157a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:35.737660 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-820a969b-2145-419d-b510-f0de54a2157a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:35.738921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-820a969b-2145-419d-b510-f0de54a2157a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:35.761447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:35.821814 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 12/23] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:35.856919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174b293-388e-4c53-a0f9-ce5061e5374f None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:35.915882 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:35.955384 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954261c-dfd7-40f8-9757-74a60bfccf8d None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:35.956028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954261c-dfd7-40f8-9757-74a60bfccf8d None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:35.956505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954261c-dfd7-40f8-9757-74a60bfccf8d None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:35.959097 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 13/24] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:35.964582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a31003-5068-4441-bf70-6067c852eec8 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:35.965142 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a31003-5068-4441-bf70-6067c852eec8 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:35.965606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a31003-5068-4441-bf70-6067c852eec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:35.992291 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174b293-388e-4c53-a0f9-ce5061e5374f None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:36.009199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-b6a31003-5068-4441-bf70-6067c852eec8 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:36.010774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b6a31003-5068-4441-bf70-6067c852eec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:36.028414 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff81c8c-5011-40cb-834d-6f53e5a7d13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:36.029408 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff81c8c-5011-40cb-834d-6f53e5a7d13f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:36.029947 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff81c8c-5011-40cb-834d-6f53e5a7d13f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:36.030428 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff81c8c-5011-40cb-834d-6f53e5a7d13f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:36.041900 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff81c8c-5011-40cb-834d-6f53e5a7d13f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:36.043675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff81c8c-5011-40cb-834d-6f53e5a7d13f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:36.060732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-6ff81c8c-5011-40cb-834d-6f53e5a7d13f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:03:36.060732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:36.060732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:36.060732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:36.060732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:36.060732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:36.060732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:36.060732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:36.060732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:36.060732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:36.060732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:36.060732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:36.060732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:03:36.060732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:03:36.060732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:03:36.061582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:03:36.061582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:36.061582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:36.061582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:36.061582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:36.061582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:36.061582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:36.061582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:36.061582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:36.061582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:36.061582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:36.061582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:36.061582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:36.061582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:36.061582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:36.061582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:36.061582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:36.062384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:03:36.062384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:03:36.062384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:03:36.062384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:03:36.062582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 12/25] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:03:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:03:36.064693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6f49e-7a3b-436a-a532-95ec84785f1b None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:36.064693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6f49e-7a3b-436a-a532-95ec84785f1b None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:36.065680 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6f49e-7a3b-436a-a532-95ec84785f1b None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:36.068703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 13/26] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:36.069602 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6f49e-7a3b-436a-a532-95ec84785f1b None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:36.104869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 14/27] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:36.106402 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6b696-47ad-456f-bdf7-2f2b11c5cbe2 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:36.113607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6b696-47ad-456f-bdf7-2f2b11c5cbe2 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:36.114060 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6b696-47ad-456f-bdf7-2f2b11c5cbe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:36.145628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-8cf6b696-47ad-456f-bdf7-2f2b11c5cbe2 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:36.147019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8cf6b696-47ad-456f-bdf7-2f2b11c5cbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:36.184258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6f49e-7a3b-436a-a532-95ec84785f1b None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:36.196285 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:36.219157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 15/28] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:36.220220 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6b696-47ad-456f-bdf7-2f2b11c5cbe2 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:36.222942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c78854-540f-4855-9a9c-6d0286c9d8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:36.223743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c78854-540f-4855-9a9c-6d0286c9d8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:36.224097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c78854-540f-4855-9a9c-6d0286c9d8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:36.224441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c78854-540f-4855-9a9c-6d0286c9d8e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:36.225350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c78854-540f-4855-9a9c-6d0286c9d8e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:36.225893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c78854-540f-4855-9a9c-6d0286c9d8e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:36.227513 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c78854-540f-4855-9a9c-6d0286c9d8e7 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:36.239786 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:36.241041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 14/29] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:36.247184 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c78854-540f-4855-9a9c-6d0286c9d8e7 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:36.325947 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6b696-47ad-456f-bdf7-2f2b11c5cbe2 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:36.387289 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c78854-540f-4855-9a9c-6d0286c9d8e7 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:36.463540 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e20f8-6e78-4866-b24d-d420ff4fad98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:36.464487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e20f8-6e78-4866-b24d-d420ff4fad98 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:36.464959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e20f8-6e78-4866-b24d-d420ff4fad98 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:36.465433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6e20f8-6e78-4866-b24d-d420ff4fad98 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:36.466294 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e20f8-6e78-4866-b24d-d420ff4fad98 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:36.468059 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6e20f8-6e78-4866-b24d-d420ff4fad98 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:36.476677 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbefed4-d896-40a3-a6c7-642559e8080b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:36.477498 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbefed4-d896-40a3-a6c7-642559e8080b None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:36.477916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbefed4-d896-40a3-a6c7-642559e8080b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:36.478323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbefed4-d896-40a3-a6c7-642559e8080b None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:36.479257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbefed4-d896-40a3-a6c7-642559e8080b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:36.480802 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbefed4-d896-40a3-a6c7-642559e8080b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:36.497020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:36.612972 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 15/30] 10.222.0.45 () {60 vars in 1141 bytes} [Thu Aug 4 02:03:36 2022] POST /identity/v3/projects => generated 301 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:03:36.614278 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e20f8-6e78-4866-b24d-d420ff4fad98 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:36.620271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 16/31] 10.222.0.45 () {60 vars in 1131 bytes} [Thu Aug 4 02:03:36 2022] POST /identity/v3/roles => generated 219 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:03:36.627250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14ffa7-4b49-4451-abee-cdead0f33f09 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:36.627706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14ffa7-4b49-4451-abee-cdead0f33f09 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:36.629116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e14ffa7-4b49-4451-abee-cdead0f33f09 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:36.635449 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 17/32] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:36 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:36.638016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402999b-73bf-42f7-ad61-1ed524bf5ee6 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:36.638302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402999b-73bf-42f7-ad61-1ed524bf5ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:36.642868 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402999b-73bf-42f7-ad61-1ed524bf5ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:36.698882 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-e402999b-73bf-42f7-ad61-1ed524bf5ee6 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:36.702826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e402999b-73bf-42f7-ad61-1ed524bf5ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:36.835601 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 18/33] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:36.863838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6e20f8-6e78-4866-b24d-d420ff4fad98 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:36.866647 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ac15b-7755-412b-a3e6-4040fae88a83 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:36.867194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ac15b-7755-412b-a3e6-4040fae88a83 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:36.867663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8ac15b-7755-412b-a3e6-4040fae88a83 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:36.869338 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 19/34] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:03:36.870227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ac15b-7755-412b-a3e6-4040fae88a83 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:36.885810 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:36.910697 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abecf9f7-d4b9-4116-a36d-7cb188d59576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:36.911369 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecf9f7-d4b9-4116-a36d-7cb188d59576 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:36.911654 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecf9f7-d4b9-4116-a36d-7cb188d59576 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:36.911929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abecf9f7-d4b9-4116-a36d-7cb188d59576 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:36.925989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abecf9f7-d4b9-4116-a36d-7cb188d59576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:03:36.933582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-abecf9f7-d4b9-4116-a36d-7cb188d59576 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:03:36.933582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:36.933582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:36.933582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:36.933582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:36.933582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:36.933582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:36.933582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:36.933582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:36.933582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:36.933582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:36.933582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:36.933582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:03:36.933582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:03:36.933582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:03:36.933582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:03:36.934476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:03:36.934476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:03:36.934476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:03:36.934476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:03:36.934476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:36.934476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:36.934476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:36.934476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:36.934476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:36.934476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:36.934476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:36.934476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:36.934476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:36.934476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:36.934476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:36.934476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:36.935476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:36.935476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:36.935476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:36.935476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:36.935476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:03:36.935476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:03:36.935476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:03:36.935476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:03:36.935476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:03:36.935476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:03:36.935476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:03:36.935476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:03:36.938113 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 16/35] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:03:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:03:36.940144 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bad55-03c9-46ac-8cab-f42d84ba7a49 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:36.940144 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bad55-03c9-46ac-8cab-f42d84ba7a49 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:36.940144 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861bad55-03c9-46ac-8cab-f42d84ba7a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:36.963045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-861bad55-03c9-46ac-8cab-f42d84ba7a49 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:36.964528 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-861bad55-03c9-46ac-8cab-f42d84ba7a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:37.018439 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8ac15b-7755-412b-a3e6-4040fae88a83 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:37.031943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:37.052664 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 17/36] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:37.052664 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bad55-03c9-46ac-8cab-f42d84ba7a49 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:37.060052 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1459b0-b8f4-44ef-b787-ac2cf1a207a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:37.060954 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1459b0-b8f4-44ef-b787-ac2cf1a207a8 None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:37.061443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1459b0-b8f4-44ef-b787-ac2cf1a207a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:37.062027 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1459b0-b8f4-44ef-b787-ac2cf1a207a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:37.063011 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1459b0-b8f4-44ef-b787-ac2cf1a207a8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:37.064772 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1459b0-b8f4-44ef-b787-ac2cf1a207a8 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:37.081337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 20/37] 10.222.0.45 () {60 vars in 1131 bytes} [Thu Aug 4 02:03:36 2022] POST /identity/v3/roles => generated 225 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:03:37.082461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1459b0-b8f4-44ef-b787-ac2cf1a207a8 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:37.165760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:37.184299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861bad55-03c9-46ac-8cab-f42d84ba7a49 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:37.220222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1459b0-b8f4-44ef-b787-ac2cf1a207a8 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:37.236279 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d737b-da1c-4e1b-9a6a-307737ae1f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:37.237560 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d737b-da1c-4e1b-9a6a-307737ae1f05 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:37.238594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d737b-da1c-4e1b-9a6a-307737ae1f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:37.239130 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d737b-da1c-4e1b-9a6a-307737ae1f05 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:37.240723 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d737b-da1c-4e1b-9a6a-307737ae1f05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:37.241301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d737b-da1c-4e1b-9a6a-307737ae1f05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:37.243528 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d737b-da1c-4e1b-9a6a-307737ae1f05 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:37.263316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 18/38] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:03:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:37.268704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d737b-da1c-4e1b-9a6a-307737ae1f05 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:37.272410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400a396-0e18-4cf8-8c16-5c0669db5496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:37.274093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400a396-0e18-4cf8-8c16-5c0669db5496 None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:37.274389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400a396-0e18-4cf8-8c16-5c0669db5496 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:37.274664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400a396-0e18-4cf8-8c16-5c0669db5496 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:37.275418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400a396-0e18-4cf8-8c16-5c0669db5496 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:37.276852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400a396-0e18-4cf8-8c16-5c0669db5496 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:37.299174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 21/39] 10.222.0.45 () {60 vars in 1131 bytes} [Thu Aug 4 02:03:37 2022] POST /identity/v3/roles => generated 223 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:03:37.341265 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:37.400336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d737b-da1c-4e1b-9a6a-307737ae1f05 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:37.439265 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ecfea-1e98-48e5-8322-d553d686a2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:37.439837 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ecfea-1e98-48e5-8322-d553d686a2e2 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:37.440116 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ecfea-1e98-48e5-8322-d553d686a2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:37.440392 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9ecfea-1e98-48e5-8322-d553d686a2e2 None admin] PATH_INFO: `/v3/users/1cdae040f69d4994bc484e82a80c2139` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:37.441733 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ecfea-1e98-48e5-8322-d553d686a2e2 None admin] RBAC: Authorizing `identity:get_user(user_id=1cdae040f69d4994bc484e82a80c2139)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:37.443209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9ecfea-1e98-48e5-8322-d553d686a2e2 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:37.445113 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 19/40] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:03:37 2022] GET /identity/v3/users/1cdae040f69d4994bc484e82a80c2139 => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:03:37.451622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400a396-0e18-4cf8-8c16-5c0669db5496 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:37.556897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400a396-0e18-4cf8-8c16-5c0669db5496 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:37.581350 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:37.590741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbfb1f-acb3-4008-9a6b-821a8a067b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:37.591386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbfb1f-acb3-4008-9a6b-821a8a067b98 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:37.591667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbfb1f-acb3-4008-9a6b-821a8a067b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:37.591941 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbfb1f-acb3-4008-9a6b-821a8a067b98 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:37.593117 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbfb1f-acb3-4008-9a6b-821a8a067b98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:37.594578 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbfb1f-acb3-4008-9a6b-821a8a067b98 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:37.596156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 22/41] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:03:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:37.602355 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ecfea-1e98-48e5-8322-d553d686a2e2 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:37.638783 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd969d9-2071-464f-8234-32bf30518774 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:37.639065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd969d9-2071-464f-8234-32bf30518774 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:37.639342 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd969d9-2071-464f-8234-32bf30518774 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:37.640719 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 23/42] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:37.646086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df4d80-d8cd-4495-9fe7-f0d58eb10aeb None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:37.646378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df4d80-d8cd-4495-9fe7-f0d58eb10aeb None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:37.646656 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df4d80-d8cd-4495-9fe7-f0d58eb10aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:37.666551 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-36df4d80-d8cd-4495-9fe7-f0d58eb10aeb None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:37.667752 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-36df4d80-d8cd-4495-9fe7-f0d58eb10aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:37.704713 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9ecfea-1e98-48e5-8322-d553d686a2e2 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:37.722222 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:37.736468 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 24/43] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:03:37.739273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a7fe6-e50f-44e1-8ee8-9f983485afab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:37.739940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a7fe6-e50f-44e1-8ee8-9f983485afab None admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:37.740220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a7fe6-e50f-44e1-8ee8-9f983485afab None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:37.740497 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8a7fe6-e50f-44e1-8ee8-9f983485afab None admin] PATH_INFO: `/v3/domains/default/users/1cdae040f69d4994bc484e82a80c2139/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:37.742459 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a7fe6-e50f-44e1-8ee8-9f983485afab None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1cdae040f69d4994bc484e82a80c2139, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:37.743907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8a7fe6-e50f-44e1-8ee8-9f983485afab None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:37.757597 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 20/44] 10.222.0.45 () {58 vars in 1361 bytes} [Thu Aug 4 02:03:37 2022] PUT /identity/v3/domains/default/users/1cdae040f69d4994bc484e82a80c2139/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:03:37.760331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df4d80-d8cd-4495-9fe7-f0d58eb10aeb None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:37.868248 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df4d80-d8cd-4495-9fe7-f0d58eb10aeb None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:37.896687 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:37.908765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59cf58c-cbde-4564-abf5-c0f299ec38fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:37.909430 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cf58c-cbde-4564-abf5-c0f299ec38fa None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:37.909763 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cf58c-cbde-4564-abf5-c0f299ec38fa None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:37.910527 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59cf58c-cbde-4564-abf5-c0f299ec38fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:37.911722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59cf58c-cbde-4564-abf5-c0f299ec38fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:37.913220 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59cf58c-cbde-4564-abf5-c0f299ec38fa None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:37.914932 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 25/45] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:03:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:37.920863 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a7fe6-e50f-44e1-8ee8-9f983485afab None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:38.023142 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8a7fe6-e50f-44e1-8ee8-9f983485afab None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:38.056842 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10e2ba-1042-4c88-889d-11a78a3f9854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:38.057455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10e2ba-1042-4c88-889d-11a78a3f9854 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:38.057739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10e2ba-1042-4c88-889d-11a78a3f9854 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:38.058010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10e2ba-1042-4c88-889d-11a78a3f9854 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:38.058708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10e2ba-1042-4c88-889d-11a78a3f9854 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:38.060419 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10e2ba-1042-4c88-889d-11a78a3f9854 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:38.087644 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 21/46] 10.222.0.45 () {60 vars in 1141 bytes} [Thu Aug 4 02:03:37 2022] POST /identity/v3/projects => generated 312 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:03:39.755788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d80c5-e7e5-4a82-901b-180c9766f995 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:39.756294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d80c5-e7e5-4a82-901b-180c9766f995 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:39.756375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d80c5-e7e5-4a82-901b-180c9766f995 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:39.757727 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 26/47] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:39.763042 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b25527-0e2b-443a-bc29-8a594aba6d72 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:39.763444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b25527-0e2b-443a-bc29-8a594aba6d72 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:39.763671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b25527-0e2b-443a-bc29-8a594aba6d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:39.781867 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-47b25527-0e2b-443a-bc29-8a594aba6d72 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:39.783083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-47b25527-0e2b-443a-bc29-8a594aba6d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:39.806428 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:39.850058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 22/48] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:39.872021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d80c5-e7e5-4a82-901b-180c9766f995 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:39.975592 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d80c5-e7e5-4a82-901b-180c9766f995 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:40.008381 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c4ad0-9a66-45ca-84bc-6f67ae2bef3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:40.009051 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c4ad0-9a66-45ca-84bc-6f67ae2bef3e None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:40.009339 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c4ad0-9a66-45ca-84bc-6f67ae2bef3e None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:40.009609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c4ad0-9a66-45ca-84bc-6f67ae2bef3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:40.020528 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c4ad0-9a66-45ca-84bc-6f67ae2bef3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:03:40.025512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-196c4ad0-9a66-45ca-84bc-6f67ae2bef3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:03:40.025512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:40.025512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:40.025512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:40.025512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:40.025512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:40.025512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:40.025512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:40.025512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:40.025512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:40.025512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:40.025512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:40.025512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:03:40.025512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:03:40.025512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:03:40.025512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:03:40.026526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:03:40.026526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:03:40.026526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:03:40.026526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:03:40.026526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:40.026526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:40.026526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:40.026526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:40.026526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:40.026526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:40.026526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:40.026526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:40.026526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:40.026526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:40.026526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:40.026526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:40.027591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:40.027591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:40.027591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:40.027591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:40.027591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:03:40.027591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:03:40.027591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:03:40.027591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:03:40.027591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:03:40.027591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:03:40.027591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:03:40.027591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:03:40.028763 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 27/49] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:03:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:03:40.032725 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b25527-0e2b-443a-bc29-8a594aba6d72 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:40.122106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddd791-ceb4-4ff4-9ca5-2595b1570638 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:40.122389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddd791-ceb4-4ff4-9ca5-2595b1570638 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:40.122674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fddd791-ceb4-4ff4-9ca5-2595b1570638 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:40.124022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 28/50] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:03:40.129007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b12af8-8774-4bd8-aa67-4fc61f213d24 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:40.129288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b12af8-8774-4bd8-aa67-4fc61f213d24 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:40.129566 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b12af8-8774-4bd8-aa67-4fc61f213d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:40.134265 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b25527-0e2b-443a-bc29-8a594aba6d72 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:40.147604 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-10b12af8-8774-4bd8-aa67-4fc61f213d24 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:40.148923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-10b12af8-8774-4bd8-aa67-4fc61f213d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:40.172394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09cb67-f247-4d14-b191-a889e0730834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:40.173032 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09cb67-f247-4d14-b191-a889e0730834 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:40.173317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09cb67-f247-4d14-b191-a889e0730834 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:40.173590 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09cb67-f247-4d14-b191-a889e0730834 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:40.174302 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09cb67-f247-4d14-b191-a889e0730834 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:40.174606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09cb67-f247-4d14-b191-a889e0730834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:40.175985 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09cb67-f247-4d14-b191-a889e0730834 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:40.184055 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 23/51] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:03:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:40.189944 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09cb67-f247-4d14-b191-a889e0730834 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:40.197473 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:40.214201 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:40.218363 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 29/52] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:03:40.241514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b12af8-8774-4bd8-aa67-4fc61f213d24 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:40.292905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09cb67-f247-4d14-b191-a889e0730834 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:40.325901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20845363-b04c-4a76-a922-a1b1ef8f8f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:40.326547 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20845363-b04c-4a76-a922-a1b1ef8f8f42 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:40.326840 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20845363-b04c-4a76-a922-a1b1ef8f8f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:40.327107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20845363-b04c-4a76-a922-a1b1ef8f8f42 None admin] PATH_INFO: `/v3/users/1cdae040f69d4994bc484e82a80c2139` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:40.328298 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20845363-b04c-4a76-a922-a1b1ef8f8f42 None admin] RBAC: Authorizing `identity:get_user(user_id=1cdae040f69d4994bc484e82a80c2139)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:40.329785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20845363-b04c-4a76-a922-a1b1ef8f8f42 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:40.332141 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 24/53] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:03:40 2022] GET /identity/v3/users/1cdae040f69d4994bc484e82a80c2139 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:03:40.337333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20845363-b04c-4a76-a922-a1b1ef8f8f42 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:40.345397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b12af8-8774-4bd8-aa67-4fc61f213d24 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:40.380372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08586eb0-f49f-49f3-a840-9ab7d9d94a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:40.381042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08586eb0-f49f-49f3-a840-9ab7d9d94a9d None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:40.381325 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08586eb0-f49f-49f3-a840-9ab7d9d94a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:40.381601 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08586eb0-f49f-49f3-a840-9ab7d9d94a9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:40.382777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08586eb0-f49f-49f3-a840-9ab7d9d94a9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:40.384232 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08586eb0-f49f-49f3-a840-9ab7d9d94a9d None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:40.385852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 30/54] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:03:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:03:40.391304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08586eb0-f49f-49f3-a840-9ab7d9d94a9d None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:40.431786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:40.441523 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:40.441972 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20845363-b04c-4a76-a922-a1b1ef8f8f42 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:40.474595 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7366e-d438-4ed2-b1a4-53be4a281083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:40.475233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7366e-d438-4ed2-b1a4-53be4a281083 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:40.475513 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7366e-d438-4ed2-b1a4-53be4a281083 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:40.475790 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7366e-d438-4ed2-b1a4-53be4a281083 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:40.476956 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7366e-d438-4ed2-b1a4-53be4a281083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:40.478446 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e7366e-d438-4ed2-b1a4-53be4a281083 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:40.480020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 25/55] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:03:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:03:40.485719 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7366e-d438-4ed2-b1a4-53be4a281083 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:40.491743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08586eb0-f49f-49f3-a840-9ab7d9d94a9d None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:40.524673 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8c17ff-c79e-40eb-a208-4065240e3d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:40.525300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c17ff-c79e-40eb-a208-4065240e3d13 None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:40.525589 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c17ff-c79e-40eb-a208-4065240e3d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:40.525865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8c17ff-c79e-40eb-a208-4065240e3d13 None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:40.526554 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c17ff-c79e-40eb-a208-4065240e3d13 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:40.527983 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8c17ff-c79e-40eb-a208-4065240e3d13 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:40.565412 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 31/56] 10.222.0.45 () {60 vars in 1140 bytes} [Thu Aug 4 02:03:40 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:03:40.586633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7366e-d438-4ed2-b1a4-53be4a281083 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:40.624378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e85af97-c81b-472f-a9bd-98d926fe8f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:40.625044 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85af97-c81b-472f-a9bd-98d926fe8f86 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:40.625348 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85af97-c81b-472f-a9bd-98d926fe8f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:40.625609 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e85af97-c81b-472f-a9bd-98d926fe8f86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:40.626429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85af97-c81b-472f-a9bd-98d926fe8f86 None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.domain.id=default, user.id=1cdae040f69d4994bc484e82a80c2139) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:40.626738 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85af97-c81b-472f-a9bd-98d926fe8f86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:40.628201 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e85af97-c81b-472f-a9bd-98d926fe8f86 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:40.636612 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 26/57] 10.222.0.45 () {56 vars in 1322 bytes} [Thu Aug 4 02:03:40 2022] GET /identity/v3/role_assignments?user.id=1cdae040f69d4994bc484e82a80c2139&scope.domain.id=default&role.id=35205394c8af4c52a61b12f92ba16644 => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:42.605890 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84fa68-476c-451a-8324-029124bd7811 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:42.606406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84fa68-476c-451a-8324-029124bd7811 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:42.606491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db84fa68-476c-451a-8324-029124bd7811 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:42.607791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 32/58] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:42.612942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdc432-7ac3-4b6b-88d7-1d5f36590088 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:42.613351 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdc432-7ac3-4b6b-88d7-1d5f36590088 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:42.613502 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fdc432-7ac3-4b6b-88d7-1d5f36590088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:42.631682 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-47fdc432-7ac3-4b6b-88d7-1d5f36590088 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:42.632913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-47fdc432-7ac3-4b6b-88d7-1d5f36590088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:42.648921 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:42.699022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 27/59] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:42.723358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84fa68-476c-451a-8324-029124bd7811 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:42.822177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db84fa68-476c-451a-8324-029124bd7811 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:42.823786 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:42.860460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf828b-34db-48e4-b6b4-e2fe5edfbb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:42.861052 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf828b-34db-48e4-b6b4-e2fe5edfbb57 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:42.861335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf828b-34db-48e4-b6b4-e2fe5edfbb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:42.861608 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf828b-34db-48e4-b6b4-e2fe5edfbb57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:42.862914 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf828b-34db-48e4-b6b4-e2fe5edfbb57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:42.864924 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf828b-34db-48e4-b6b4-e2fe5edfbb57 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:42.866517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 33/60] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:03:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:42.872298 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdc432-7ac3-4b6b-88d7-1d5f36590088 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:42.970623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fdc432-7ac3-4b6b-88d7-1d5f36590088 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:43.006043 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34143c07-f84b-4d8e-9562-041457f042e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:43.006659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34143c07-f84b-4d8e-9562-041457f042e7 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:43.006941 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34143c07-f84b-4d8e-9562-041457f042e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:43.007243 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34143c07-f84b-4d8e-9562-041457f042e7 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:43.007932 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34143c07-f84b-4d8e-9562-041457f042e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:43.009418 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34143c07-f84b-4d8e-9562-041457f042e7 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:43.036875 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 28/61] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:03:42 2022] POST /identity/v3/users => generated 275 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:03:46.131358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab2777-9d53-4cc8-a224-540eb133fbbf None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:46.132390 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab2777-9d53-4cc8-a224-540eb133fbbf None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:46.132925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab2777-9d53-4cc8-a224-540eb133fbbf None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:46.134496 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 34/62] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:46.139916 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f27924-474e-490b-8edf-01a7333395b1 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:46.140789 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f27924-474e-490b-8edf-01a7333395b1 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:46.141304 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f27924-474e-490b-8edf-01a7333395b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:46.169647 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-f1f27924-474e-490b-8edf-01a7333395b1 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:46.171186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-f1f27924-474e-490b-8edf-01a7333395b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:46.173576 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:46.256728 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 29/63] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:46.294278 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab2777-9d53-4cc8-a224-540eb133fbbf None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:46.399425 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0270632-cf80-426d-94d1-d12bd90810be None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:46.399715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0270632-cf80-426d-94d1-d12bd90810be None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:46.399912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0270632-cf80-426d-94d1-d12bd90810be None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:46.401315 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 30/64] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:03:46.415744 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd31860-73af-465d-a40e-30f699a75d55 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:46.416050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd31860-73af-465d-a40e-30f699a75d55 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:46.416303 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd31860-73af-465d-a40e-30f699a75d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:46.443858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-3cd31860-73af-465d-a40e-30f699a75d55 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:46.445120 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-3cd31860-73af-465d-a40e-30f699a75d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:46.474293 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ab2777-9d53-4cc8-a224-540eb133fbbf None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:46.531270 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635de836-a14c-4ba2-965e-eb99e9608a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:46.532435 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635de836-a14c-4ba2-965e-eb99e9608a12 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:46.532899 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635de836-a14c-4ba2-965e-eb99e9608a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:46.533478 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635de836-a14c-4ba2-965e-eb99e9608a12 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:46.548579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635de836-a14c-4ba2-965e-eb99e9608a12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:03:46.553490 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 31/65] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:03:46.555631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-635de836-a14c-4ba2-965e-eb99e9608a12 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:03:46.555631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:46.555631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:46.555631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:46.555631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:46.555631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:46.555631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:46.555631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:46.555631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:46.555631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:46.555631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:46.555631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:46.555631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:03:46.555631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:03:46.555631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:03:46.555631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:03:46.556676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:03:46.556676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:03:46.556676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:03:46.556676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:03:46.556676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:46.556676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:46.556676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:46.556676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:46.556676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:46.556676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:46.556676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:46.556676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:46.556676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:46.556676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:46.556676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:46.556676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:46.557607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:46.557607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:46.557607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:46.557607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:46.557607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:03:46.557607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:03:46.557607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:03:46.557607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:03:46.557607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:03:46.557607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:03:46.557607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:03:46.557607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:03:46.559523 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 35/66] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:03:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:03:46.564881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5e451-9901-4e5d-a4d5-cfd3f011b34f None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:46.564881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5e451-9901-4e5d-a4d5-cfd3f011b34f None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:46.564881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5e451-9901-4e5d-a4d5-cfd3f011b34f None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:46.564881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 32/67] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:46 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:46.566191 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635de836-a14c-4ba2-965e-eb99e9608a12 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:46.570094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff72d2-4211-426e-be89-6cc9fcfda189 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:46.570391 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff72d2-4211-426e-be89-6cc9fcfda189 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:46.570675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff72d2-4211-426e-be89-6cc9fcfda189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:46.590940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-29ff72d2-4211-426e-be89-6cc9fcfda189 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:46.592125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-29ff72d2-4211-426e-be89-6cc9fcfda189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:46.619732 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:46.669447 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:46.676574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 33/68] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:03:46.685747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff72d2-4211-426e-be89-6cc9fcfda189 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:46.718974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635de836-a14c-4ba2-965e-eb99e9608a12 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:46.752488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e578a-c730-4467-aa32-12269e9f404d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:46.753146 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e578a-c730-4467-aa32-12269e9f404d None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:46.753468 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e578a-c730-4467-aa32-12269e9f404d None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:46.753746 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e578a-c730-4467-aa32-12269e9f404d None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:46.754466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e578a-c730-4467-aa32-12269e9f404d None admin] RBAC: Adding query filter params (name=member) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:46.754825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e578a-c730-4467-aa32-12269e9f404d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:46.756223 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e578a-c730-4467-aa32-12269e9f404d None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:46.765952 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 36/69] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:03:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:46.767135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382e64b-eba6-430d-99c7-835557782fb0 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:46.767415 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382e64b-eba6-430d-99c7-835557782fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:46.767684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382e64b-eba6-430d-99c7-835557782fb0 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:46.768912 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 37/70] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:46.770113 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382e64b-eba6-430d-99c7-835557782fb0 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:46.794823 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff72d2-4211-426e-be89-6cc9fcfda189 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:46.827857 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92498a46-a609-4d6b-a056-6ed01b7d0335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:46.828734 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92498a46-a609-4d6b-a056-6ed01b7d0335 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:46.829023 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92498a46-a609-4d6b-a056-6ed01b7d0335 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:46.829302 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92498a46-a609-4d6b-a056-6ed01b7d0335 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:46.840011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92498a46-a609-4d6b-a056-6ed01b7d0335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:03:46.844971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-92498a46-a609-4d6b-a056-6ed01b7d0335 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:03:46.844971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:46.844971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:46.844971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:46.844971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:46.844971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:46.844971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:46.844971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:46.844971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:46.844971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:46.844971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:46.844971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:46.844971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:03:46.844971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:03:46.844971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:03:46.844971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:03:46.845824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:03:46.845824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:03:46.845824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:03:46.845824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:03:46.845824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:46.845824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:46.845824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:46.845824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:46.845824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:46.845824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:46.845824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:46.845824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:46.845824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:46.845824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:46.845824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:46.845824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:46.846569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:46.846569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:46.846569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:46.846569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:46.846569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:03:46.846569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:03:46.846569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:03:46.846569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:03:46.846569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:03:46.846569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:03:46.846569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:03:46.846569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:03:46.846569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 34/71] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:03:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:03:46.848297 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00caf9be-8972-4004-8ba5-91750bcb3782 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:46.848581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00caf9be-8972-4004-8ba5-91750bcb3782 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:46.848889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00caf9be-8972-4004-8ba5-91750bcb3782 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:46.850153 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 35/72] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:46.850862 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00caf9be-8972-4004-8ba5-91750bcb3782 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:46.867188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:46.877515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382e64b-eba6-430d-99c7-835557782fb0 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:46.911394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dea7aa5-627d-4ae3-b9c9-28175802b459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:46.911952 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea7aa5-627d-4ae3-b9c9-28175802b459 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:46.912236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea7aa5-627d-4ae3-b9c9-28175802b459 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:46.912521 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dea7aa5-627d-4ae3-b9c9-28175802b459 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:46.914758 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:46.922672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dea7aa5-627d-4ae3-b9c9-28175802b459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:03:46.928130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-9dea7aa5-627d-4ae3-b9c9-28175802b459 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:03:46.928130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:46.928130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:46.928130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:46.928130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:46.928130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:46.928130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:46.928130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:46.928130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:46.928130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:46.928130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:46.928130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:46.928130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:03:46.928130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:03:46.928130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:03:46.928130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:03:46.929297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:03:46.929297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:03:46.929297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:03:46.929297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:03:46.929297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:46.929297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:46.929297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:46.929297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:46.929297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:46.929297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:46.929297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:46.929297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:46.929297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:46.929297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:46.929297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:46.929297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:46.936454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:46.936454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:46.936454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:46.936454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:46.936454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:03:46.936454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:03:46.936454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:03:46.936454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:03:46.936454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:03:46.936454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:03:46.936454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:03:46.936454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:03:46.936454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 38/73] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:03:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:03:46.936454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b995d45-cbc4-47c1-95c1-603063b40902 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:46.937229 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b995d45-cbc4-47c1-95c1-603063b40902 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:46.937229 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b995d45-cbc4-47c1-95c1-603063b40902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:46.949765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-8b995d45-cbc4-47c1-95c1-603063b40902 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:46.951030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8b995d45-cbc4-47c1-95c1-603063b40902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:46.960279 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00caf9be-8972-4004-8ba5-91750bcb3782 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:46.993088 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac44a4-185d-40b9-a97f-159195f997b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:46.993723 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac44a4-185d-40b9-a97f-159195f997b6 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:46.994011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac44a4-185d-40b9-a97f-159195f997b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:46.994287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac44a4-185d-40b9-a97f-159195f997b6 None admin] PATH_INFO: `/v3/users/e88786550f274e61877748f90084c3f4` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:47.008574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac44a4-185d-40b9-a97f-159195f997b6 None admin] RBAC: Authorizing `identity:get_user(user_id=e88786550f274e61877748f90084c3f4)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:47.010088 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac44a4-185d-40b9-a97f-159195f997b6 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:47.011927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 36/74] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:03:46 2022] GET /identity/v3/users/e88786550f274e61877748f90084c3f4 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:47.012577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac44a4-185d-40b9-a97f-159195f997b6 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:47.020759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 39/75] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:47.022009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4c9f0-ee12-4f70-8475-057698dafc73 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:47.022300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4c9f0-ee12-4f70-8475-057698dafc73 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:47.022579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4c9f0-ee12-4f70-8475-057698dafc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:47.039806 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-9ed4c9f0-ee12-4f70-8475-057698dafc73 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:47.041035 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9ed4c9f0-ee12-4f70-8475-057698dafc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:47.108441 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 40/76] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:47.109390 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4c9f0-ee12-4f70-8475-057698dafc73 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:47.115266 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac44a4-185d-40b9-a97f-159195f997b6 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:47.127846 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:47.135268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:47.153463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1a26f-4fa1-482c-96c7-4d789aed0915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:47.154196 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1a26f-4fa1-482c-96c7-4d789aed0915 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:47.154487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1a26f-4fa1-482c-96c7-4d789aed0915 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:47.154765 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1a26f-4fa1-482c-96c7-4d789aed0915 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:47.155485 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1a26f-4fa1-482c-96c7-4d789aed0915 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:47.155832 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1a26f-4fa1-482c-96c7-4d789aed0915 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:47.157251 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1a26f-4fa1-482c-96c7-4d789aed0915 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:47.168129 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 37/77] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:03:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:47.168931 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1a26f-4fa1-482c-96c7-4d789aed0915 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:47.220210 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4c9f0-ee12-4f70-8475-057698dafc73 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:47.254977 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fe5a4-a22f-4448-b39d-5a2ad883b9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:47.255592 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fe5a4-a22f-4448-b39d-5a2ad883b9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:47.255876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fe5a4-a22f-4448-b39d-5a2ad883b9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:47.256167 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5fe5a4-a22f-4448-b39d-5a2ad883b9b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:47.256888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fe5a4-a22f-4448-b39d-5a2ad883b9b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:47.257191 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fe5a4-a22f-4448-b39d-5a2ad883b9b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:47.258554 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5fe5a4-a22f-4448-b39d-5a2ad883b9b4 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:47.266333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 41/78] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:03:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:47.267123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fe5a4-a22f-4448-b39d-5a2ad883b9b4 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:47.270300 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1a26f-4fa1-482c-96c7-4d789aed0915 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:47.305012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449e19d-19bf-4235-a48b-14e5b2b1c79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:47.305638 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449e19d-19bf-4235-a48b-14e5b2b1c79f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:47.305916 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449e19d-19bf-4235-a48b-14e5b2b1c79f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:47.306195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7449e19d-19bf-4235-a48b-14e5b2b1c79f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:47.345117 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:47.351463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:47.371963 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5fe5a4-a22f-4448-b39d-5a2ad883b9b4 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:47.373805 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7449e19d-19bf-4235-a48b-14e5b2b1c79f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:47.375287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7449e19d-19bf-4235-a48b-14e5b2b1c79f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:47.404733 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67acf5e-3f1a-443e-95b9-e0f085a8518c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:47.405373 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67acf5e-3f1a-443e-95b9-e0f085a8518c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:47.405661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67acf5e-3f1a-443e-95b9-e0f085a8518c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:47.405940 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67acf5e-3f1a-443e-95b9-e0f085a8518c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:47.407132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67acf5e-3f1a-443e-95b9-e0f085a8518c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:47.408586 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67acf5e-3f1a-443e-95b9-e0f085a8518c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:47.410860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 42/79] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:03:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:47.411624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67acf5e-3f1a-443e-95b9-e0f085a8518c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:47.476195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 38/80] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:03:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:03:47.477057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449e19d-19bf-4235-a48b-14e5b2b1c79f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:47.511186 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67acf5e-3f1a-443e-95b9-e0f085a8518c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:47.546262 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e659e56-472f-4ff9-b2b3-8a59c3eff585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:47.546903 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e659e56-472f-4ff9-b2b3-8a59c3eff585 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:47.547181 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e659e56-472f-4ff9-b2b3-8a59c3eff585 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:47.547457 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e659e56-472f-4ff9-b2b3-8a59c3eff585 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:47.554190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:47.557597 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e659e56-472f-4ff9-b2b3-8a59c3eff585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:03:47.563127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-4e659e56-472f-4ff9-b2b3-8a59c3eff585 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:03:47.563127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:47.563127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:47.563127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:47.563127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:47.563127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:47.563127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:47.563127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:47.563127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:47.563127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:47.563127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:47.563127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:47.563127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:03:47.563127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:03:47.563127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:03:47.563127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:03:47.564070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:03:47.564070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:03:47.564070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:03:47.564070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:03:47.564070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:47.564070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:47.564070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:47.564070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:47.564070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:47.564070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:47.564070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:47.564070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:47.564070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:47.564070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:47.564070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:47.564070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:47.564886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:47.564886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:47.564886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:47.564886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:47.564886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:03:47.564886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:03:47.564886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:03:47.564886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:03:47.564886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:03:47.564886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:03:47.564886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:03:47.564886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:03:47.564886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 43/81] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:03:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:03:47.565503 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e659e56-472f-4ff9-b2b3-8a59c3eff585 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:47.569894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:47.578566 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7449e19d-19bf-4235-a48b-14e5b2b1c79f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:47.613812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afcb10-670f-4163-861c-8a602287f0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:47.614455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afcb10-670f-4163-861c-8a602287f0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:47.614819 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afcb10-670f-4163-861c-8a602287f0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:47.615009 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7afcb10-670f-4163-861c-8a602287f0d3 None admin] PATH_INFO: `/v3/users/1cdae040f69d4994bc484e82a80c2139` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:47.616169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7afcb10-670f-4163-861c-8a602287f0d3 None admin] RBAC: Authorizing `identity:get_user(user_id=1cdae040f69d4994bc484e82a80c2139)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:47.617653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7afcb10-670f-4163-861c-8a602287f0d3 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:47.619250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 39/82] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:03:47 2022] GET /identity/v3/users/1cdae040f69d4994bc484e82a80c2139 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:47.619983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afcb10-670f-4163-861c-8a602287f0d3 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:47.670610 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e659e56-472f-4ff9-b2b3-8a59c3eff585 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:47.704631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354754b-1333-49ad-89a6-945b79fcebfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:47.705301 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354754b-1333-49ad-89a6-945b79fcebfd None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:47.705582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354754b-1333-49ad-89a6-945b79fcebfd None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:47.705854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b354754b-1333-49ad-89a6-945b79fcebfd None admin] PATH_INFO: `/v3/users/e88786550f274e61877748f90084c3f4` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:47.707544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b354754b-1333-49ad-89a6-945b79fcebfd None admin] RBAC: Authorizing `identity:get_user(user_id=e88786550f274e61877748f90084c3f4)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:47.709030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b354754b-1333-49ad-89a6-945b79fcebfd None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:47.710646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 44/83] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:03:47 2022] GET /identity/v3/users/e88786550f274e61877748f90084c3f4 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:47.711406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354754b-1333-49ad-89a6-945b79fcebfd None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:47.727781 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7afcb10-670f-4163-861c-8a602287f0d3 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:47.761852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ab38e-a2c6-4297-a2c6-0c6c167a78e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:47.762502 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ab38e-a2c6-4297-a2c6-0c6c167a78e7 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:47.762787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ab38e-a2c6-4297-a2c6-0c6c167a78e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:47.763072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2ab38e-a2c6-4297-a2c6-0c6c167a78e7 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:47.763760 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ab38e-a2c6-4297-a2c6-0c6c167a78e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:47.765281 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2ab38e-a2c6-4297-a2c6-0c6c167a78e7 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:47.782610 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:47.787243 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 40/84] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:03:47 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:03:47.788357 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ab38e-a2c6-4297-a2c6-0c6c167a78e7 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:47.790888 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:47.830294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b354754b-1333-49ad-89a6-945b79fcebfd None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:47.870912 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990db70-3ddb-44ad-9d99-bfa273a1c6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:47.871674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990db70-3ddb-44ad-9d99-bfa273a1c6cf None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:47.871953 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990db70-3ddb-44ad-9d99-bfa273a1c6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:47.872232 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990db70-3ddb-44ad-9d99-bfa273a1c6cf None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:47.873517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990db70-3ddb-44ad-9d99-bfa273a1c6cf None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:47.875008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990db70-3ddb-44ad-9d99-bfa273a1c6cf None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:47.876745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 45/85] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:03:47 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:47.877548 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990db70-3ddb-44ad-9d99-bfa273a1c6cf None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:47.915577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2ab38e-a2c6-4297-a2c6-0c6c167a78e7 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:47.955941 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62043df9-af11-4817-ba35-dd8518f46311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:47.956655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62043df9-af11-4817-ba35-dd8518f46311 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:47.956938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62043df9-af11-4817-ba35-dd8518f46311 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:47.957241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62043df9-af11-4817-ba35-dd8518f46311 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:47.957955 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62043df9-af11-4817-ba35-dd8518f46311 None admin] RBAC: Adding query filter params (name=member) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:47.958263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62043df9-af11-4817-ba35-dd8518f46311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:47.959679 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62043df9-af11-4817-ba35-dd8518f46311 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:47.970581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 41/86] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:03:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:47.971739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62043df9-af11-4817-ba35-dd8518f46311 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:48.003899 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990db70-3ddb-44ad-9d99-bfa273a1c6cf None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:48.048477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91497ac-7362-4a27-8e41-b4bd054abd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:48.049155 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91497ac-7362-4a27-8e41-b4bd054abd5b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:48.049435 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91497ac-7362-4a27-8e41-b4bd054abd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:48.049708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91497ac-7362-4a27-8e41-b4bd054abd5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:48.056181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:48.061027 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:48.106252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62043df9-af11-4817-ba35-dd8518f46311 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:48.137200 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91497ac-7362-4a27-8e41-b4bd054abd5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:48.139261 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91497ac-7362-4a27-8e41-b4bd054abd5b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:48.147335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfada3-a234-4a9b-8c33-e7d227a67c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:48.147968 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfada3-a234-4a9b-8c33-e7d227a67c0f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:48.148254 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfada3-a234-4a9b-8c33-e7d227a67c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:48.148530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfada3-a234-4a9b-8c33-e7d227a67c0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:48.228410 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adfada3-a234-4a9b-8c33-e7d227a67c0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:48.230743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adfada3-a234-4a9b-8c33-e7d227a67c0f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:48.291454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 46/87] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:03:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:03:48.292246 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91497ac-7362-4a27-8e41-b4bd054abd5b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:48.301542 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:48.345244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:48.350067 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 42/88] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:03:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:03:48.351251 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfada3-a234-4a9b-8c33-e7d227a67c0f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:48.417694 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91497ac-7362-4a27-8e41-b4bd054abd5b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:48.457869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec6542-fc6d-4b81-86ca-731be68f3e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:48.458593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec6542-fc6d-4b81-86ca-731be68f3e7a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:48.458884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec6542-fc6d-4b81-86ca-731be68f3e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:48.459259 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ec6542-fc6d-4b81-86ca-731be68f3e7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:48.460064 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec6542-fc6d-4b81-86ca-731be68f3e7a None admin] RBAC: Adding query filter params (role.id=8fc75c351a0349e294deeed1ff391c4c, scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef, user.id=e88786550f274e61877748f90084c3f4) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:48.460375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec6542-fc6d-4b81-86ca-731be68f3e7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:48.462095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ec6542-fc6d-4b81-86ca-731be68f3e7a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:48.467902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfada3-a234-4a9b-8c33-e7d227a67c0f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:48.468185 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 47/89] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:03:48 2022] GET /identity/v3/role_assignments?user.id=e88786550f274e61877748f90084c3f4&scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef&role.id=8fc75c351a0349e294deeed1ff391c4c => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:48.469041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec6542-fc6d-4b81-86ca-731be68f3e7a None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:48.506841 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b11c2-7db5-40eb-8bb1-50602044cdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:48.507511 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b11c2-7db5-40eb-8bb1-50602044cdf5 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:48.507798 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b11c2-7db5-40eb-8bb1-50602044cdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:48.508076 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b11c2-7db5-40eb-8bb1-50602044cdf5 None admin] PATH_INFO: `/v3/users/e88786550f274e61877748f90084c3f4` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:48.509368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b11c2-7db5-40eb-8bb1-50602044cdf5 None admin] RBAC: Authorizing `identity:get_user(user_id=e88786550f274e61877748f90084c3f4)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:48.510865 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b11c2-7db5-40eb-8bb1-50602044cdf5 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:48.512594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 43/90] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:03:48 2022] GET /identity/v3/users/e88786550f274e61877748f90084c3f4 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:48.513423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b11c2-7db5-40eb-8bb1-50602044cdf5 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:48.545719 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:48.614694 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ec6542-fc6d-4b81-86ca-731be68f3e7a None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:48.616606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:48.628648 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b11c2-7db5-40eb-8bb1-50602044cdf5 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:48.665434 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de7a0a-4369-4f53-88f2-ef25bff46edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:48.666095 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de7a0a-4369-4f53-88f2-ef25bff46edb None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:48.666380 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de7a0a-4369-4f53-88f2-ef25bff46edb None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:48.666659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de7a0a-4369-4f53-88f2-ef25bff46edb None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:48.667878 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de7a0a-4369-4f53-88f2-ef25bff46edb None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:48.669424 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65de7a0a-4369-4f53-88f2-ef25bff46edb None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:48.671358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 44/91] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:03:48 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:48.672148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de7a0a-4369-4f53-88f2-ef25bff46edb None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:48.685410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f02c52-a2a2-4ae3-b6d2-47a09e74ffc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:48.686077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f02c52-a2a2-4ae3-b6d2-47a09e74ffc1 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:48.686456 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f02c52-a2a2-4ae3-b6d2-47a09e74ffc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:48.686737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f02c52-a2a2-4ae3-b6d2-47a09e74ffc1 None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:48.689599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f02c52-a2a2-4ae3-b6d2-47a09e74ffc1 None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:48.691219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f02c52-a2a2-4ae3-b6d2-47a09e74ffc1 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:48.693082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 48/92] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:03:48 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:48.693956 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f02c52-a2a2-4ae3-b6d2-47a09e74ffc1 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:48.780111 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de7a0a-4369-4f53-88f2-ef25bff46edb None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:48.797833 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:48.824569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800db08-94b1-4749-8b30-865286bdcaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:48.825373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800db08-94b1-4749-8b30-865286bdcaa2 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:48.825759 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800db08-94b1-4749-8b30-865286bdcaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:48.826134 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d800db08-94b1-4749-8b30-865286bdcaa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:48.900643 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f02c52-a2a2-4ae3-b6d2-47a09e74ffc1 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:48.948121 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800db08-94b1-4749-8b30-865286bdcaa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:48.949726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d800db08-94b1-4749-8b30-865286bdcaa2 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:48.966771 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2e9d9-d4f0-4b27-acfe-ba80514c18c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:48.967406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2e9d9-d4f0-4b27-acfe-ba80514c18c3 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:48.967692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2e9d9-d4f0-4b27-acfe-ba80514c18c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:48.967979 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff2e9d9-d4f0-4b27-acfe-ba80514c18c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:48.968805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff2e9d9-d4f0-4b27-acfe-ba80514c18c3 None admin] RBAC: Adding query filter params (role.id=80e5528d7c8742f78e29e64c1216176c, scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef, user.id=e88786550f274e61877748f90084c3f4) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:48.969122 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff2e9d9-d4f0-4b27-acfe-ba80514c18c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:48.970565 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff2e9d9-d4f0-4b27-acfe-ba80514c18c3 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:48.983727 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 49/93] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:03:48 2022] GET /identity/v3/role_assignments?user.id=e88786550f274e61877748f90084c3f4&scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef&role.id=80e5528d7c8742f78e29e64c1216176c => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:48.984566 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2e9d9-d4f0-4b27-acfe-ba80514c18c3 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:49.051428 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:49.061977 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:49.066406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 45/94] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:03:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:03:49.072384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800db08-94b1-4749-8b30-865286bdcaa2 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:49.125140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff2e9d9-d4f0-4b27-acfe-ba80514c18c3 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:49.164153 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5856b9f-3ba0-4e8b-8a50-429caa4b6309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:49.164811 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5856b9f-3ba0-4e8b-8a50-429caa4b6309 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:49.165098 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5856b9f-3ba0-4e8b-8a50-429caa4b6309 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:49.165374 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5856b9f-3ba0-4e8b-8a50-429caa4b6309 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:49.167354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5856b9f-3ba0-4e8b-8a50-429caa4b6309 None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef, user.id=1cdae040f69d4994bc484e82a80c2139) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:49.167768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5856b9f-3ba0-4e8b-8a50-429caa4b6309 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:49.169296 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5856b9f-3ba0-4e8b-8a50-429caa4b6309 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:49.175777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 50/95] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:03:48 2022] GET /identity/v3/role_assignments?user.id=1cdae040f69d4994bc484e82a80c2139&scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef&role.id=35205394c8af4c52a61b12f92ba16644 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:49.248963 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d800db08-94b1-4749-8b30-865286bdcaa2 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:49.297231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d790a-d561-4ad2-9dca-7b91493c30a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:49.297896 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d790a-d561-4ad2-9dca-7b91493c30a0 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:49.298180 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d790a-d561-4ad2-9dca-7b91493c30a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:49.298457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d790a-d561-4ad2-9dca-7b91493c30a0 None admin] PATH_INFO: `/v3/projects/c60d517a79da4691a934dbee19a2ad95` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:49.301152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d790a-d561-4ad2-9dca-7b91493c30a0 None admin] RBAC: Authorizing `identity:get_project(project_id=c60d517a79da4691a934dbee19a2ad95)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:49.302704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d790a-d561-4ad2-9dca-7b91493c30a0 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:49.304429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 46/96] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:03:49 2022] GET /identity/v3/projects/c60d517a79da4691a934dbee19a2ad95 => generated 312 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:49.311048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5856b9f-3ba0-4e8b-8a50-429caa4b6309 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:49.442015 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:49.446457 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5856b9f-3ba0-4e8b-8a50-429caa4b6309 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:49.495029 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231dacc-c8a8-40ce-8628-ca98250fc73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:49.496020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231dacc-c8a8-40ce-8628-ca98250fc73c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:49.496500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231dacc-c8a8-40ce-8628-ca98250fc73c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:49.496989 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231dacc-c8a8-40ce-8628-ca98250fc73c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:49.497978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231dacc-c8a8-40ce-8628-ca98250fc73c None admin] RBAC: Adding query filter params (role.id=8fc75c351a0349e294deeed1ff391c4c, scope.project.id=c60d517a79da4691a934dbee19a2ad95, user.id=e88786550f274e61877748f90084c3f4) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:49.498502 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231dacc-c8a8-40ce-8628-ca98250fc73c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:49.500129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231dacc-c8a8-40ce-8628-ca98250fc73c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:49.508830 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 51/97] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:03:49 2022] GET /identity/v3/role_assignments?user.id=e88786550f274e61877748f90084c3f4&scope.project.id=c60d517a79da4691a934dbee19a2ad95&role.id=8fc75c351a0349e294deeed1ff391c4c => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:50.324305 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27f41a-cd50-41c8-9759-73810a83883c None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:50.324872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27f41a-cd50-41c8-9759-73810a83883c None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:50.324872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27f41a-cd50-41c8-9759-73810a83883c None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:50.326372 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 47/98] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:50.331794 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a61ee7-a14b-4fe5-874f-81601c3986fc None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:50.332199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a61ee7-a14b-4fe5-874f-81601c3986fc None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:50.332383 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a61ee7-a14b-4fe5-874f-81601c3986fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:50.357576 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-62a61ee7-a14b-4fe5-874f-81601c3986fc None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:50.358778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-62a61ee7-a14b-4fe5-874f-81601c3986fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:50.450901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 52/99] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:50.486988 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27f41a-cd50-41c8-9759-73810a83883c None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:50.569939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:50.613940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27f41a-cd50-41c8-9759-73810a83883c None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:50.662818 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9084609-cf85-4e10-8fad-e165e2052d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:50.663467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9084609-cf85-4e10-8fad-e165e2052d81 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:50.663750 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9084609-cf85-4e10-8fad-e165e2052d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:50.664028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9084609-cf85-4e10-8fad-e165e2052d81 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:50.665617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9084609-cf85-4e10-8fad-e165e2052d81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:50.667124 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9084609-cf85-4e10-8fad-e165e2052d81 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:50.668947 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 48/100] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:03:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:50.674907 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a61ee7-a14b-4fe5-874f-81601c3986fc None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:50.687447 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325255a4-c341-4442-a420-9612a97b4dbd None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:50.688103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325255a4-c341-4442-a420-9612a97b4dbd None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:50.688571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325255a4-c341-4442-a420-9612a97b4dbd None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:50.690218 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 49/101] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:03:50.695863 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0e748-934d-4da0-a63f-121b673a0759 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:50.696413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0e748-934d-4da0-a63f-121b673a0759 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:50.696929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d0e748-934d-4da0-a63f-121b673a0759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:50.718213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-77d0e748-934d-4da0-a63f-121b673a0759 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:50.719799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-77d0e748-934d-4da0-a63f-121b673a0759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:50.830725 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 50/102] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:03:50.857835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0e748-934d-4da0-a63f-121b673a0759 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:50.884478 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:50.898390 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a61ee7-a14b-4fe5-874f-81601c3986fc None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:50.952957 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c280d-99f0-49ea-a0f5-078daf1aaadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:50.953739 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c280d-99f0-49ea-a0f5-078daf1aaadb None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:50.953972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c280d-99f0-49ea-a0f5-078daf1aaadb None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:50.954256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c280d-99f0-49ea-a0f5-078daf1aaadb None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:50.954952 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c280d-99f0-49ea-a0f5-078daf1aaadb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:50.956456 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c280d-99f0-49ea-a0f5-078daf1aaadb None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:50.958093 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:50.996528 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d0e748-934d-4da0-a63f-121b673a0759 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:51.010567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 53/103] 10.222.0.45 () {60 vars in 1141 bytes} [Thu Aug 4 02:03:50 2022] POST /identity/v3/projects => generated 302 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:03:51.048009 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee037ffe-c669-4867-bc5f-05fd93d73313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:51.049443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee037ffe-c669-4867-bc5f-05fd93d73313 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:51.050240 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee037ffe-c669-4867-bc5f-05fd93d73313 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:51.051622 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee037ffe-c669-4867-bc5f-05fd93d73313 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:51.066587 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee037ffe-c669-4867-bc5f-05fd93d73313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:03:51.072145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-ee037ffe-c669-4867-bc5f-05fd93d73313 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:03:51.072145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:51.072145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:51.072145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:51.072145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:51.072145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:51.072145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:51.072145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:51.072145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:51.072145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:51.072145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:51.072145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:51.072145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:03:51.072145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:03:51.072145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:03:51.072145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:03:51.073013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:03:51.073013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:03:51.073013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:03:51.073013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:03:51.073013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:51.073013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:51.073013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:51.073013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:51.073013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:51.073013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:51.073013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:51.073013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:51.073013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:51.073013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:51.073013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:51.073013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:51.073776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:51.073776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:51.073776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:51.073776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:51.073776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:03:51.073776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:03:51.073776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:03:51.073776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:03:51.073776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:03:51.073776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:03:51.073776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:03:51.073776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:03:51.080968 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 51/104] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:03:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:03:51.082669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c280d-99f0-49ea-a0f5-078daf1aaadb None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:51.234079 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c280d-99f0-49ea-a0f5-078daf1aaadb None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:51.279242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d831323-34c8-4e03-a3c2-ea2c7d0aee62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:51.279883 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d831323-34c8-4e03-a3c2-ea2c7d0aee62 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:51.280169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d831323-34c8-4e03-a3c2-ea2c7d0aee62 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:51.280446 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d831323-34c8-4e03-a3c2-ea2c7d0aee62 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:51.281207 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d831323-34c8-4e03-a3c2-ea2c7d0aee62 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:51.281518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d831323-34c8-4e03-a3c2-ea2c7d0aee62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:51.282898 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d831323-34c8-4e03-a3c2-ea2c7d0aee62 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:51.293226 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 54/105] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:03:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:51.299013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee037ffe-c669-4867-bc5f-05fd93d73313 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:51.417482 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee037ffe-c669-4867-bc5f-05fd93d73313 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:51.445422 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:51.454844 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f376417-bec4-49fa-8c73-88230542ff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:51.455834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f376417-bec4-49fa-8c73-88230542ff1e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:51.456324 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f376417-bec4-49fa-8c73-88230542ff1e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:51.456875 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f376417-bec4-49fa-8c73-88230542ff1e None admin] PATH_INFO: `/v3/users/e38c92e5198e45c3bb8ef525b2fec7b9` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:51.471441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f376417-bec4-49fa-8c73-88230542ff1e None admin] RBAC: Authorizing `identity:get_user(user_id=e38c92e5198e45c3bb8ef525b2fec7b9)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:51.473353 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f376417-bec4-49fa-8c73-88230542ff1e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:51.475336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 52/106] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:03:51 2022] GET /identity/v3/users/e38c92e5198e45c3bb8ef525b2fec7b9 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:51.485935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d831323-34c8-4e03-a3c2-ea2c7d0aee62 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:51.499852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:51.623391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d831323-34c8-4e03-a3c2-ea2c7d0aee62 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:51.676220 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e38ab9-bf02-4606-bd50-e725fc59444e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:51.676892 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e38ab9-bf02-4606-bd50-e725fc59444e None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:51.677179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e38ab9-bf02-4606-bd50-e725fc59444e None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:51.677459 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e38ab9-bf02-4606-bd50-e725fc59444e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:51.763115 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e38ab9-bf02-4606-bd50-e725fc59444e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:51.764515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e38ab9-bf02-4606-bd50-e725fc59444e None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:51.773513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:51.811092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062901e0-1ee0-49fc-a0f8-568275766efe None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:51.811729 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062901e0-1ee0-49fc-a0f8-568275766efe None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:51.812214 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062901e0-1ee0-49fc-a0f8-568275766efe None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:51.815164 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 53/107] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:51.820095 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a83d4b-3891-4e58-84fb-7f0de4a167cb None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:51.820647 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a83d4b-3891-4e58-84fb-7f0de4a167cb None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:51.821146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a83d4b-3891-4e58-84fb-7f0de4a167cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:51.854414 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-74a83d4b-3891-4e58-84fb-7f0de4a167cb None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:51.856214 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-74a83d4b-3891-4e58-84fb-7f0de4a167cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:51.896069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 55/108] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:03:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:03:51.900298 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e38ab9-bf02-4606-bd50-e725fc59444e None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:51.976733 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 54/109] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:03:52.046614 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a83d4b-3891-4e58-84fb-7f0de4a167cb None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:52.076170 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e38ab9-bf02-4606-bd50-e725fc59444e None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:52.087710 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:52.143703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db80cbbd-3bfb-48f6-a742-7e43013fec92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:52.144510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80cbbd-3bfb-48f6-a742-7e43013fec92 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:52.144832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80cbbd-3bfb-48f6-a742-7e43013fec92 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:52.145117 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db80cbbd-3bfb-48f6-a742-7e43013fec92 None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:52.148414 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80cbbd-3bfb-48f6-a742-7e43013fec92 None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:52.149965 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db80cbbd-3bfb-48f6-a742-7e43013fec92 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:52.151676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 56/110] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:03:51 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:03:52.156690 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6e149-7425-4a64-901f-316595bee358 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:52.156690 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6e149-7425-4a64-901f-316595bee358 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:52.156690 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6e149-7425-4a64-901f-316595bee358 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:52.156690 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 57/111] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:52.161357 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660e217-2418-44bf-ade3-8b446fbaf96f None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:52.161649 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660e217-2418-44bf-ade3-8b446fbaf96f None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:52.161932 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660e217-2418-44bf-ade3-8b446fbaf96f None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:52.163353 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 58/112] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:52.164060 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660e217-2418-44bf-ade3-8b446fbaf96f None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:52.192436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a83d4b-3891-4e58-84fb-7f0de4a167cb None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:52.213284 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:52.228997 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f109fb3-3758-41d7-8ad0-2069992d6459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:52.229363 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f109fb3-3758-41d7-8ad0-2069992d6459 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:52.229641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f109fb3-3758-41d7-8ad0-2069992d6459 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:52.229975 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f109fb3-3758-41d7-8ad0-2069992d6459 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:52.241759 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f109fb3-3758-41d7-8ad0-2069992d6459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:03:52.246902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-7f109fb3-3758-41d7-8ad0-2069992d6459 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:03:52.246902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:52.246902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:52.246902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:52.246902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:52.246902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:52.246902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:52.246902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:52.246902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:52.246902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:52.246902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:52.246902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:52.246902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:03:52.246902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:03:52.246902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:03:52.246902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:03:52.247771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:03:52.247771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:03:52.247771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:03:52.247771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:03:52.247771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:52.247771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:52.247771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:52.247771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:52.247771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:52.247771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:52.247771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:52.247771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:52.247771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:52.247771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:52.247771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:52.247771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:52.248533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:52.248533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:52.248533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:52.248533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:52.248533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:03:52.248533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:03:52.248533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:03:52.248533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:03:52.248533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:03:52.248533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:03:52.248533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:03:52.248533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:03:52.248533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 55/113] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:03:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:03:52.254654 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f5c65-d889-4a92-85d3-179b732dba45 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:52.254968 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f5c65-d889-4a92-85d3-179b732dba45 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:52.256287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8f5c65-d889-4a92-85d3-179b732dba45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:52.276320 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-ac8f5c65-d889-4a92-85d3-179b732dba45 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:52.277522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ac8f5c65-d889-4a92-85d3-179b732dba45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:52.317938 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660e217-2418-44bf-ade3-8b446fbaf96f None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:52.354681 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 56/114] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:52.356028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cdcb1-add3-4114-8288-e618555c253d None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:52.356542 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cdcb1-add3-4114-8288-e618555c253d None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:52.356667 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938cdcb1-add3-4114-8288-e618555c253d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:52.378059 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe2a64-5baf-4f22-9785-3205ecb160d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:52.379300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe2a64-5baf-4f22-9785-3205ecb160d4 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:52.379586 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe2a64-5baf-4f22-9785-3205ecb160d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:52.384653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffe2a64-5baf-4f22-9785-3205ecb160d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:52.384653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe2a64-5baf-4f22-9785-3205ecb160d4 None admin] RBAC: Adding query filter params (role.id=9e8771d0146c45e6afaf7915e215ac91, scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef, user.id=e38c92e5198e45c3bb8ef525b2fec7b9) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:52.384653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe2a64-5baf-4f22-9785-3205ecb160d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:52.384653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffe2a64-5baf-4f22-9785-3205ecb160d4 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:52.385120 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-938cdcb1-add3-4114-8288-e618555c253d None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:52.385120 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-938cdcb1-add3-4114-8288-e618555c253d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:52.391663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 59/115] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:03:52 2022] GET /identity/v3/role_assignments?user.id=e38c92e5198e45c3bb8ef525b2fec7b9&scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef&role.id=9e8771d0146c45e6afaf7915e215ac91 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:52.392709 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe2a64-5baf-4f22-9785-3205ecb160d4 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:52.461103 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:52.538704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 57/116] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:52.539491 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cdcb1-add3-4114-8288-e618555c253d None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:52.576226 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffe2a64-5baf-4f22-9785-3205ecb160d4 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:52.604415 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:52.621435 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ca99c-4ded-45be-8e66-88e66c03aca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:52.622213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ca99c-4ded-45be-8e66-88e66c03aca5 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:52.622503 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ca99c-4ded-45be-8e66-88e66c03aca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:52.622779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6ca99c-4ded-45be-8e66-88e66c03aca5 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:52.623510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ca99c-4ded-45be-8e66-88e66c03aca5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:52.623820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ca99c-4ded-45be-8e66-88e66c03aca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:52.625519 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6ca99c-4ded-45be-8e66-88e66c03aca5 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:52.637601 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 60/117] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:03:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:52.647162 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ca99c-4ded-45be-8e66-88e66c03aca5 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:52.710353 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938cdcb1-add3-4114-8288-e618555c253d None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:52.749104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76631bc6-2034-4b63-ad5c-bcd52c81e219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:52.749886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76631bc6-2034-4b63-ad5c-bcd52c81e219 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:52.750173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76631bc6-2034-4b63-ad5c-bcd52c81e219 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:52.750451 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76631bc6-2034-4b63-ad5c-bcd52c81e219 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:52.762551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76631bc6-2034-4b63-ad5c-bcd52c81e219 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:03:52.767950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-76631bc6-2034-4b63-ad5c-bcd52c81e219 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:03:52.767950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:52.767950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:52.767950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:52.767950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:52.767950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:52.767950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:52.767950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:52.767950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:52.767950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:52.767950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:52.767950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:52.767950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:03:52.767950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:03:52.767950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:03:52.767950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:03:52.768951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:03:52.768951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:03:52.768951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:03:52.768951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:03:52.768951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:52.768951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:52.768951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:52.768951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:52.768951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:52.768951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:52.768951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:52.768951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:52.768951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:52.768951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:52.768951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:52.768951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:52.769717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:52.769717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:52.769717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:52.769717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:52.769717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:03:52.769717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:03:52.769717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:03:52.769717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:03:52.769717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:03:52.769717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:03:52.769717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:03:52.769717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:03:52.770287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:52.770354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 58/118] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:03:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:03:52.772666 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76631bc6-2034-4b63-ad5c-bcd52c81e219 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:52.777705 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6ca99c-4ded-45be-8e66-88e66c03aca5 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:52.813418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493ff9c-d025-4bd1-ac94-132b4453adda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:52.814068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493ff9c-d025-4bd1-ac94-132b4453adda None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:52.814344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493ff9c-d025-4bd1-ac94-132b4453adda None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:52.814618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3493ff9c-d025-4bd1-ac94-132b4453adda None admin] PATH_INFO: `/v3/roles/member` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:52.826039 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3493ff9c-d025-4bd1-ac94-132b4453adda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:03:52.831460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-3493ff9c-d025-4bd1-ac94-132b4453adda None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:03:52.831460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:52.831460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:52.831460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:52.831460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:52.831460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:52.831460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:52.831460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:52.831460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:52.831460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:52.831460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:52.831460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:52.831460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:03:52.831460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:03:52.831460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:03:52.831460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:03:52.833495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:03:52.833495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:03:52.833495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:03:52.833495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:03:52.833495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:52.833495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:52.833495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:52.833495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:52.833495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:52.833495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:52.833495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:52.833495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:52.833495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:52.833495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:52.833495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:52.833495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:52.835399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:52.835399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:52.835399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:52.835399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:52.835399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:03:52.835399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:03:52.835399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:03:52.835399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:03:52.835399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:03:52.835399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:03:52.835399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:03:52.835399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:03:52.835399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 61/119] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:03:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:03:52.835399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493ff9c-d025-4bd1-ac94-132b4453adda None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:52.920726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:52.929739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76631bc6-2034-4b63-ad5c-bcd52c81e219 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:52.935493 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3493ff9c-d025-4bd1-ac94-132b4453adda None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:52.966634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd693e1-2865-494e-af34-07ca4166a1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:52.967284 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd693e1-2865-494e-af34-07ca4166a1cf None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:52.967569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd693e1-2865-494e-af34-07ca4166a1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:52.967848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd693e1-2865-494e-af34-07ca4166a1cf None admin] PATH_INFO: `/v3/users/e88786550f274e61877748f90084c3f4` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:52.969153 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd693e1-2865-494e-af34-07ca4166a1cf None admin] RBAC: Authorizing `identity:get_user(user_id=e88786550f274e61877748f90084c3f4)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:52.970624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd693e1-2865-494e-af34-07ca4166a1cf None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:52.971228 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf59de-f586-4b00-8a78-f79d46c40c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:52.971856 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf59de-f586-4b00-8a78-f79d46c40c49 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:52.972178 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf59de-f586-4b00-8a78-f79d46c40c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:52.972605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 59/120] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:03:52 2022] GET /identity/v3/users/e88786550f274e61877748f90084c3f4 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:52.972784 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf59de-f586-4b00-8a78-f79d46c40c49 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:52.973211 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf59de-f586-4b00-8a78-f79d46c40c49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:52.973521 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf59de-f586-4b00-8a78-f79d46c40c49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:52.974895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf59de-f586-4b00-8a78-f79d46c40c49 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:52.977564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd693e1-2865-494e-af34-07ca4166a1cf None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:52.983934 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 62/121] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:03:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:52.989146 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021efe8-c8fa-4edd-94c4-f52233ded74b None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:52.989428 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021efe8-c8fa-4edd-94c4-f52233ded74b None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:52.989703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021efe8-c8fa-4edd-94c4-f52233ded74b None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:52.991054 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 63/122] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:52.991798 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021efe8-c8fa-4edd-94c4-f52233ded74b None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:53.025051 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:53.091286 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd693e1-2865-494e-af34-07ca4166a1cf None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:53.102939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021efe8-c8fa-4edd-94c4-f52233ded74b None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:53.128423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029eb70f-d926-4fcb-bfcd-03dc61d541ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:53.129089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eb70f-d926-4fcb-bfcd-03dc61d541ca None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:53.129376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eb70f-d926-4fcb-bfcd-03dc61d541ca None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:53.129771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eb70f-d926-4fcb-bfcd-03dc61d541ca None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:53.130555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029eb70f-d926-4fcb-bfcd-03dc61d541ca None admin] RBAC: Adding query filter params (name=member) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:53.130864 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029eb70f-d926-4fcb-bfcd-03dc61d541ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:53.132306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029eb70f-d926-4fcb-bfcd-03dc61d541ca None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:53.137691 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dddb62b-50eb-4f6b-adad-b6f57f19b041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:53.138388 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddb62b-50eb-4f6b-adad-b6f57f19b041 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:53.138675 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddb62b-50eb-4f6b-adad-b6f57f19b041 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:53.138947 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddb62b-50eb-4f6b-adad-b6f57f19b041 None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:53.140146 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dddb62b-50eb-4f6b-adad-b6f57f19b041 None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:53.141793 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 60/123] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:03:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:53.141969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dddb62b-50eb-4f6b-adad-b6f57f19b041 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:53.142748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029eb70f-d926-4fcb-bfcd-03dc61d541ca None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:53.143430 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 64/124] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:03:52 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:53.144580 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86730625-6fb6-4e36-8c22-cfc6e2dc3847 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:53.144888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86730625-6fb6-4e36-8c22-cfc6e2dc3847 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:53.145164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86730625-6fb6-4e36-8c22-cfc6e2dc3847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:53.163133 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-86730625-6fb6-4e36-8c22-cfc6e2dc3847 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:53.164315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-86730625-6fb6-4e36-8c22-cfc6e2dc3847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:53.176108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:53.237031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 65/125] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:53.238675 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86730625-6fb6-4e36-8c22-cfc6e2dc3847 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:53.267894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029eb70f-d926-4fcb-bfcd-03dc61d541ca None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:53.271417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:53.313389 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ef7df-1a70-464e-a38d-3cb36258df97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:53.314197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ef7df-1a70-464e-a38d-3cb36258df97 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:53.314485 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ef7df-1a70-464e-a38d-3cb36258df97 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:53.315073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ef7df-1a70-464e-a38d-3cb36258df97 None admin] PATH_INFO: `/v3/users/1cdae040f69d4994bc484e82a80c2139` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:53.316564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ef7df-1a70-464e-a38d-3cb36258df97 None admin] RBAC: Authorizing `identity:get_user(user_id=1cdae040f69d4994bc484e82a80c2139)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:53.318203 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ef7df-1a70-464e-a38d-3cb36258df97 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:53.320470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 61/126] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:03:53 2022] GET /identity/v3/users/1cdae040f69d4994bc484e82a80c2139 => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:53.321551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ef7df-1a70-464e-a38d-3cb36258df97 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:53.358822 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86730625-6fb6-4e36-8c22-cfc6e2dc3847 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:53.396236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4a49c-4e02-43c6-a718-7a0215d15b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:53.397051 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4a49c-4e02-43c6-a718-7a0215d15b1b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:53.397390 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4a49c-4e02-43c6-a718-7a0215d15b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:53.397673 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b4a49c-4e02-43c6-a718-7a0215d15b1b None admin] PATH_INFO: `/v3/users/e88786550f274e61877748f90084c3f4` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:53.398900 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4a49c-4e02-43c6-a718-7a0215d15b1b None admin] RBAC: Authorizing `identity:get_user(user_id=e88786550f274e61877748f90084c3f4)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:53.400462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b4a49c-4e02-43c6-a718-7a0215d15b1b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:53.402540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 66/127] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:03:53 2022] GET /identity/v3/users/e88786550f274e61877748f90084c3f4 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:53.403142 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4a49c-4e02-43c6-a718-7a0215d15b1b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:53.432389 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:53.443647 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ef7df-1a70-464e-a38d-3cb36258df97 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:53.485832 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a692e-cd49-4314-bae7-77840deff391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:53.486514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a692e-cd49-4314-bae7-77840deff391 None admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:53.486804 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a692e-cd49-4314-bae7-77840deff391 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:53.487079 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a692e-cd49-4314-bae7-77840deff391 None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef/users/e88786550f274e61877748f90084c3f4/roles/80e5528d7c8742f78e29e64c1216176c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:53.489869 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a692e-cd49-4314-bae7-77840deff391 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f4f29d6d3a14c9eb0650231a82bbdef, user_id=e88786550f274e61877748f90084c3f4, role_id=80e5528d7c8742f78e29e64c1216176c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:53.491889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a692e-cd49-4314-bae7-77840deff391 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:53.502568 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:53.511179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 62/128] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:03:53 2022] PUT /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef/users/e88786550f274e61877748f90084c3f4/roles/80e5528d7c8742f78e29e64c1216176c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:03:53.513491 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a692e-cd49-4314-bae7-77840deff391 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:53.525238 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b4a49c-4e02-43c6-a718-7a0215d15b1b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:53.599418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad826945-a79b-470f-a580-eb87bbf97c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:53.602864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad826945-a79b-470f-a580-eb87bbf97c9d None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:53.604405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad826945-a79b-470f-a580-eb87bbf97c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:53.604905 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad826945-a79b-470f-a580-eb87bbf97c9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:53.629014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad826945-a79b-470f-a580-eb87bbf97c9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:03:53.637777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-ad826945-a79b-470f-a580-eb87bbf97c9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:03:53.637777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:53.637777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:53.637777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:53.637777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:53.637777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:53.637777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:53.637777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:53.637777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:53.637777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:53.637777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:53.637777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:53.637777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:03:53.637777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:03:53.637777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:03:53.637777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:03:53.638736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:03:53.638736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:03:53.638736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:03:53.638736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:03:53.638736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:53.638736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:53.638736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:53.638736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:53.638736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:53.638736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:53.638736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:53.638736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:53.638736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:53.638736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:53.638736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:53.638736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:53.639487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:53.639487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:53.639487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:53.639487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:53.639487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:03:53.639487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:03:53.639487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:03:53.639487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:03:53.639487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:03:53.639487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:03:53.639487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:03:53.639487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:03:53.643701 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 67/129] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:03:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:03:53.646040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad826945-a79b-470f-a580-eb87bbf97c9d None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:53.741377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a692e-cd49-4314-bae7-77840deff391 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:53.780588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc94bf-ac35-4596-a6d9-8a2af3dafa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:53.781274 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc94bf-ac35-4596-a6d9-8a2af3dafa1c None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:53.781557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc94bf-ac35-4596-a6d9-8a2af3dafa1c None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:53.781657 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad826945-a79b-470f-a580-eb87bbf97c9d None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:53.781835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dc94bf-ac35-4596-a6d9-8a2af3dafa1c None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:53.783150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc94bf-ac35-4596-a6d9-8a2af3dafa1c None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:53.785427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dc94bf-ac35-4596-a6d9-8a2af3dafa1c None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:53.787135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 63/130] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:03:53 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:53.787912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc94bf-ac35-4596-a6d9-8a2af3dafa1c None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:53.813687 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:53.824654 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bcb41-ca09-4772-bd60-e8dc633decab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:53.825312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bcb41-ca09-4772-bd60-e8dc633decab None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:53.825595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bcb41-ca09-4772-bd60-e8dc633decab None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:53.825869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585bcb41-ca09-4772-bd60-e8dc633decab None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:53.827787 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585bcb41-ca09-4772-bd60-e8dc633decab None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:53.829294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585bcb41-ca09-4772-bd60-e8dc633decab None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:53.831179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 68/131] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:03:53 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:53.831881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bcb41-ca09-4772-bd60-e8dc633decab None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:53.858528 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:53.901039 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dc94bf-ac35-4596-a6d9-8a2af3dafa1c None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:53.938064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccd87f-360c-4a74-ac04-a588e2b25d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:53.938836 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccd87f-360c-4a74-ac04-a588e2b25d00 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:53.939375 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccd87f-360c-4a74-ac04-a588e2b25d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:53.939754 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ccd87f-360c-4a74-ac04-a588e2b25d00 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:53.940791 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ccd87f-360c-4a74-ac04-a588e2b25d00 None admin] RBAC: Adding query filter params (name=member) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:53.941202 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ccd87f-360c-4a74-ac04-a588e2b25d00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:53.942801 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ccd87f-360c-4a74-ac04-a588e2b25d00 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:53.952169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 64/132] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:03:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:53.953192 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccd87f-360c-4a74-ac04-a588e2b25d00 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:54.025136 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585bcb41-ca09-4772-bd60-e8dc633decab None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:54.041029 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:54.060401 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ccd87f-360c-4a74-ac04-a588e2b25d00 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:54.096205 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cabf7a-b2fe-4e0a-8217-69bf5d54a29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:54.097581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cabf7a-b2fe-4e0a-8217-69bf5d54a29a None admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:54.098010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cabf7a-b2fe-4e0a-8217-69bf5d54a29a None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:54.098492 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cabf7a-b2fe-4e0a-8217-69bf5d54a29a None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef/users/e88786550f274e61877748f90084c3f4/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:54.100608 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cabf7a-b2fe-4e0a-8217-69bf5d54a29a None admin] RBAC: Authorizing `identity:create_grant(project_id=2f4f29d6d3a14c9eb0650231a82bbdef, user_id=e88786550f274e61877748f90084c3f4, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:54.102265 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cabf7a-b2fe-4e0a-8217-69bf5d54a29a None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:54.122282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 65/133] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:03:53 2022] PUT /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef/users/e88786550f274e61877748f90084c3f4/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:03:54.123111 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cabf7a-b2fe-4e0a-8217-69bf5d54a29a None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:54.123868 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ce8b1-71b9-4670-993a-7195fa205e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:54.124561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ce8b1-71b9-4670-993a-7195fa205e21 None admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:54.124873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ce8b1-71b9-4670-993a-7195fa205e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:54.125155 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ce8b1-71b9-4670-993a-7195fa205e21 None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef/users/1cdae040f69d4994bc484e82a80c2139/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:54.127296 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ce8b1-71b9-4670-993a-7195fa205e21 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f4f29d6d3a14c9eb0650231a82bbdef, user_id=1cdae040f69d4994bc484e82a80c2139, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:54.128947 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ce8b1-71b9-4670-993a-7195fa205e21 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:54.147515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 69/134] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:03:53 2022] PUT /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef/users/1cdae040f69d4994bc484e82a80c2139/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:03:54.149010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bbcf1-ac6a-4ca4-8574-244a26565120 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:54.149301 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bbcf1-ac6a-4ca4-8574-244a26565120 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:54.149582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826bbcf1-ac6a-4ca4-8574-244a26565120 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:54.155552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 70/135] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:54.157537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4903d-e03c-4f70-9ab7-1f21e4304e6c None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:54.157842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4903d-e03c-4f70-9ab7-1f21e4304e6c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:54.158116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4903d-e03c-4f70-9ab7-1f21e4304e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:54.180664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-fac4903d-e03c-4f70-9ab7-1f21e4304e6c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:54.181962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-fac4903d-e03c-4f70-9ab7-1f21e4304e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:54.291961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cabf7a-b2fe-4e0a-8217-69bf5d54a29a None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:54.306167 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 71/136] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:03:54.345759 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595ba00-0f7b-42cf-b2e8-615f43908729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:54.346439 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595ba00-0f7b-42cf-b2e8-615f43908729 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:54.346733 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595ba00-0f7b-42cf-b2e8-615f43908729 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:54.347066 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595ba00-0f7b-42cf-b2e8-615f43908729 None admin] PATH_INFO: `/v3/users/e88786550f274e61877748f90084c3f4` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:54.353605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595ba00-0f7b-42cf-b2e8-615f43908729 None admin] RBAC: Authorizing `identity:get_user(user_id=e88786550f274e61877748f90084c3f4)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:54.355138 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595ba00-0f7b-42cf-b2e8-615f43908729 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:54.357392 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 66/137] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:03:54 2022] GET /identity/v3/users/e88786550f274e61877748f90084c3f4 => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:54.362638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac4903d-e03c-4f70-9ab7-1f21e4304e6c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:54.372416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:54.384042 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595ba00-0f7b-42cf-b2e8-615f43908729 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:54.400879 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:54.501581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595ba00-0f7b-42cf-b2e8-615f43908729 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:54.516414 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac4903d-e03c-4f70-9ab7-1f21e4304e6c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:54.538863 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c9122-9e67-41db-8b3c-63fd947e598b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:54.539585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c9122-9e67-41db-8b3c-63fd947e598b None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:54.539876 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c9122-9e67-41db-8b3c-63fd947e598b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:54.540170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c9122-9e67-41db-8b3c-63fd947e598b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:54.541595 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c9122-9e67-41db-8b3c-63fd947e598b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:54.543139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c9122-9e67-41db-8b3c-63fd947e598b None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:54.546021 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 67/138] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:03:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:03:54.552416 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c9122-9e67-41db-8b3c-63fd947e598b None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:54.584808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e34df-8b99-4f52-964a-3763791acd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:54.585702 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e34df-8b99-4f52-964a-3763791acd44 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:54.586146 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e34df-8b99-4f52-964a-3763791acd44 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:54.586613 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478e34df-8b99-4f52-964a-3763791acd44 None admin] PATH_INFO: `/v3/projects/c60d517a79da4691a934dbee19a2ad95` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:54.590908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e34df-8b99-4f52-964a-3763791acd44 None admin] RBAC: Authorizing `identity:get_project(project_id=c60d517a79da4691a934dbee19a2ad95)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:54.592635 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478e34df-8b99-4f52-964a-3763791acd44 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:54.594504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 72/139] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:03:54 2022] GET /identity/v3/projects/c60d517a79da4691a934dbee19a2ad95 => generated 312 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:54.600841 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e34df-8b99-4f52-964a-3763791acd44 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:54.641008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:54.667068 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c9122-9e67-41db-8b3c-63fd947e598b None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:54.705851 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117939b-8ce6-494c-a031-a4fad2007638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:54.706531 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117939b-8ce6-494c-a031-a4fad2007638 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:54.706859 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117939b-8ce6-494c-a031-a4fad2007638 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:54.707323 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117939b-8ce6-494c-a031-a4fad2007638 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:54.708073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117939b-8ce6-494c-a031-a4fad2007638 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:54.709682 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117939b-8ce6-494c-a031-a4fad2007638 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:54.733463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 68/140] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:03:54 2022] POST /identity/v3/users => generated 283 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:03:54.746209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:54.785033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478e34df-8b99-4f52-964a-3763791acd44 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:54.832208 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c4256-d57a-4575-8ef4-8cd1913bce0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:54.832985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c4256-d57a-4575-8ef4-8cd1913bce0c None admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:54.833268 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c4256-d57a-4575-8ef4-8cd1913bce0c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:54.833543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c4256-d57a-4575-8ef4-8cd1913bce0c None admin] PATH_INFO: `/v3/projects/c60d517a79da4691a934dbee19a2ad95/users/e88786550f274e61877748f90084c3f4/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:54.840847 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c4256-d57a-4575-8ef4-8cd1913bce0c None admin] RBAC: Authorizing `identity:create_grant(project_id=c60d517a79da4691a934dbee19a2ad95, user_id=e88786550f274e61877748f90084c3f4, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:54.842485 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c4256-d57a-4575-8ef4-8cd1913bce0c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:54.866316 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 73/141] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:03:54 2022] PUT /identity/v3/projects/c60d517a79da4691a934dbee19a2ad95/users/e88786550f274e61877748f90084c3f4/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:03:55.193388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd1377-f90e-4219-9b4d-ce1170871e26 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:55.194318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd1377-f90e-4219-9b4d-ce1170871e26 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:55.194598 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd1377-f90e-4219-9b4d-ce1170871e26 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:55.195980 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 69/142] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:55.201624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e79b5-f1f7-4e2f-baa7-32454b7506ea None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:55.201913 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e79b5-f1f7-4e2f-baa7-32454b7506ea None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:55.202190 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e79b5-f1f7-4e2f-baa7-32454b7506ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:55.232116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-535e79b5-f1f7-4e2f-baa7-32454b7506ea None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:55.233356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-535e79b5-f1f7-4e2f-baa7-32454b7506ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:55.328141 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 74/143] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:55.360879 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd1377-f90e-4219-9b4d-ce1170871e26 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:55.499163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd1377-f90e-4219-9b4d-ce1170871e26 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:55.543843 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ccfd8-bea3-4b08-b065-acfba4ae2c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:55.544492 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ccfd8-bea3-4b08-b065-acfba4ae2c9c None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:55.544855 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ccfd8-bea3-4b08-b065-acfba4ae2c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:55.545139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10ccfd8-bea3-4b08-b065-acfba4ae2c9c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:55.547103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:55.560746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10ccfd8-bea3-4b08-b065-acfba4ae2c9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:03:55.567103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-d10ccfd8-bea3-4b08-b065-acfba4ae2c9c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:03:55.567103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:55.567103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:55.567103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:55.567103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:55.567103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:55.567103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:55.567103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:55.567103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:55.567103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:55.567103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:55.567103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:55.567103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:03:55.567103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:03:55.567103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:03:55.567103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:03:55.568008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:03:55.568008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:03:55.568008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:03:55.568008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:03:55.568008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:55.568008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:55.568008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:55.568008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:55.568008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:55.568008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:55.568008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:55.568008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:55.568008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:55.568008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:55.568008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:55.568008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:55.568794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:55.568794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:55.568794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:55.568794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:55.568794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:03:55.568794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:03:55.568794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:03:55.568794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:03:55.568794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:03:55.568794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:03:55.568794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:03:55.568794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:03:55.568794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 70/144] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:03:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:03:55.576706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e79b5-f1f7-4e2f-baa7-32454b7506ea None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:55.686242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:55.731688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e79b5-f1f7-4e2f-baa7-32454b7506ea None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:55.768951 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabf64f-c539-47b3-b545-ea5ab2eb3c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:55.769502 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabf64f-c539-47b3-b545-ea5ab2eb3c7f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:55.769786 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabf64f-c539-47b3-b545-ea5ab2eb3c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:55.770062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dabf64f-c539-47b3-b545-ea5ab2eb3c7f None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:55.770787 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabf64f-c539-47b3-b545-ea5ab2eb3c7f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:55.771093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabf64f-c539-47b3-b545-ea5ab2eb3c7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:55.772477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dabf64f-c539-47b3-b545-ea5ab2eb3c7f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:55.784667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 75/145] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:03:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:55.790127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ccfd8-bea3-4b08-b065-acfba4ae2c9c None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:55.936838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10ccfd8-bea3-4b08-b065-acfba4ae2c9c None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:55.979883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3c083-5f3e-4d51-8538-1af4c3a1193b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:55.980547 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3c083-5f3e-4d51-8538-1af4c3a1193b None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:55.980857 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3c083-5f3e-4d51-8538-1af4c3a1193b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:55.981138 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3c083-5f3e-4d51-8538-1af4c3a1193b None admin] PATH_INFO: `/v3/users/e38c92e5198e45c3bb8ef525b2fec7b9` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:55.983970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3c083-5f3e-4d51-8538-1af4c3a1193b None admin] RBAC: Authorizing `identity:get_user(user_id=e38c92e5198e45c3bb8ef525b2fec7b9)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:55.985476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3c083-5f3e-4d51-8538-1af4c3a1193b None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:55.987116 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 71/146] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:03:55 2022] GET /identity/v3/users/e38c92e5198e45c3bb8ef525b2fec7b9 => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:55.993001 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabf64f-c539-47b3-b545-ea5ab2eb3c7f None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:56.117085 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dabf64f-c539-47b3-b545-ea5ab2eb3c7f None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:56.148949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:56.158782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff49c21-1a62-43ca-96a2-07957f55d3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:56.159785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff49c21-1a62-43ca-96a2-07957f55d3f6 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:56.160261 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff49c21-1a62-43ca-96a2-07957f55d3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:56.160776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff49c21-1a62-43ca-96a2-07957f55d3f6 None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:56.162198 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff49c21-1a62-43ca-96a2-07957f55d3f6 None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:56.163903 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff49c21-1a62-43ca-96a2-07957f55d3f6 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:56.165816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 76/147] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:03:55 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:56.171650 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3c083-5f3e-4d51-8538-1af4c3a1193b None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:56.243233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:56.298780 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3c083-5f3e-4d51-8538-1af4c3a1193b None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:56.336853 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341d8435-0bb1-49c3-a610-0864fa9e3ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:56.337520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d8435-0bb1-49c3-a610-0864fa9e3ea1 None admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:56.337798 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d8435-0bb1-49c3-a610-0864fa9e3ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:56.338073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341d8435-0bb1-49c3-a610-0864fa9e3ea1 None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef/users/e38c92e5198e45c3bb8ef525b2fec7b9/roles/9e8771d0146c45e6afaf7915e215ac91` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:56.346166 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341d8435-0bb1-49c3-a610-0864fa9e3ea1 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f4f29d6d3a14c9eb0650231a82bbdef, user_id=e38c92e5198e45c3bb8ef525b2fec7b9, role_id=9e8771d0146c45e6afaf7915e215ac91)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:56.348187 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341d8435-0bb1-49c3-a610-0864fa9e3ea1 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:56.370452 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 72/148] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:03:56 2022] PUT /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef/users/e38c92e5198e45c3bb8ef525b2fec7b9/roles/9e8771d0146c45e6afaf7915e215ac91 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:03:56.714643 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38206f-34c8-4538-807e-19eb6d4dffd0 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:56.715170 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38206f-34c8-4538-807e-19eb6d4dffd0 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:56.715245 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38206f-34c8-4538-807e-19eb6d4dffd0 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:56.716897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 77/149] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:56.742535 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1564c4-34df-4552-9d6a-c91d38d35dd9 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:56.742816 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1564c4-34df-4552-9d6a-c91d38d35dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:56.743082 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1564c4-34df-4552-9d6a-c91d38d35dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:56.768317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-dc1564c4-34df-4552-9d6a-c91d38d35dd9 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:56.769607 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-dc1564c4-34df-4552-9d6a-c91d38d35dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:56.850961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 73/150] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:56.899770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38206f-34c8-4538-807e-19eb6d4dffd0 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:57.021100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38206f-34c8-4538-807e-19eb6d4dffd0 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:57.064939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e26fc-f810-4b9a-89cc-8d8a886899ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:57.065588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e26fc-f810-4b9a-89cc-8d8a886899ea None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:57.066405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e26fc-f810-4b9a-89cc-8d8a886899ea None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:57.066681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e26fc-f810-4b9a-89cc-8d8a886899ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:57.080785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e26fc-f810-4b9a-89cc-8d8a886899ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:03:57.086888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-f72e26fc-f810-4b9a-89cc-8d8a886899ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:03:57.086888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:57.086888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:57.086888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:57.086888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:57.086888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:57.086888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:57.086888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:57.086888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:57.086888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:57.086888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:57.086888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:57.086888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:03:57.086888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:03:57.086888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:03:57.086888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:03:57.087791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:03:57.087791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:03:57.087791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:03:57.087791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:03:57.087791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:57.087791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:57.087791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:57.087791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:57.087791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:57.087791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:57.087791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:57.087791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:57.087791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:57.087791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:57.087791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:57.087791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:57.088591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:57.088591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:57.088591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:57.088591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:57.088591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:03:57.088591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:03:57.088591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:03:57.088591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:03:57.088591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:03:57.088591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:03:57.088591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:03:57.088591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:03:57.091613 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 78/151] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:03:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:03:57.093880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1564c4-34df-4552-9d6a-c91d38d35dd9 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:57.157206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:57.219587 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1564c4-34df-4552-9d6a-c91d38d35dd9 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:57.264325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ab112-22b4-4d1c-a1de-345993ed0e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:57.265033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ab112-22b4-4d1c-a1de-345993ed0e68 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:57.265308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ab112-22b4-4d1c-a1de-345993ed0e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:57.265585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4ab112-22b4-4d1c-a1de-345993ed0e68 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:57.266374 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ab112-22b4-4d1c-a1de-345993ed0e68 None admin] RBAC: Adding query filter params (name=member) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:57.266688 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ab112-22b4-4d1c-a1de-345993ed0e68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:57.268103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4ab112-22b4-4d1c-a1de-345993ed0e68 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:57.277503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 74/152] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:03:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:57.283053 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e26fc-f810-4b9a-89cc-8d8a886899ea None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:57.346840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:57.398714 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e26fc-f810-4b9a-89cc-8d8a886899ea None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:57.441330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e342e8-4e76-4795-a894-e3889c766dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:57.441985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e342e8-4e76-4795-a894-e3889c766dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:57.442266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e342e8-4e76-4795-a894-e3889c766dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:57.442544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e342e8-4e76-4795-a894-e3889c766dd8 None admin] PATH_INFO: `/v3/users/e88786550f274e61877748f90084c3f4` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:57.448774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e342e8-4e76-4795-a894-e3889c766dd8 None admin] RBAC: Authorizing `identity:get_user(user_id=e88786550f274e61877748f90084c3f4)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:57.450324 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e342e8-4e76-4795-a894-e3889c766dd8 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:57.452200 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 79/153] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:03:57 2022] GET /identity/v3/users/e88786550f274e61877748f90084c3f4 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:57.457518 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ab112-22b4-4d1c-a1de-345993ed0e68 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:57.548719 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e58af4-0fef-4e35-97b2-ce38292ad697 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:57.548903 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e58af4-0fef-4e35-97b2-ce38292ad697 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:57.549181 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e58af4-0fef-4e35-97b2-ce38292ad697 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:57.551121 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 80/154] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:57.566583 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b94d6-8624-4cbf-80bf-a7c7345cc80b None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:57.566894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b94d6-8624-4cbf-80bf-a7c7345cc80b None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:57.567126 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b94d6-8624-4cbf-80bf-a7c7345cc80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:57.586215 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4ab112-22b4-4d1c-a1de-345993ed0e68 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:57.601775 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-702b94d6-8624-4cbf-80bf-a7c7345cc80b None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:57.605163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-702b94d6-8624-4cbf-80bf-a7c7345cc80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:57.625539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:57.682240 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65fc27-a584-494b-8a8e-c7c38d55351b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:57.682874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65fc27-a584-494b-8a8e-c7c38d55351b None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:57.683163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65fc27-a584-494b-8a8e-c7c38d55351b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:57.683436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d65fc27-a584-494b-8a8e-c7c38d55351b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:57.723495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 81/155] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:57.724586 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd67fe-8115-4e8d-a701-2f7ae74b0744 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:57.725051 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd67fe-8115-4e8d-a701-2f7ae74b0744 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:57.725339 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd67fe-8115-4e8d-a701-2f7ae74b0744 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:57.734280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 82/156] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:57.741095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0d371-834c-4096-895b-0ff62e516ee6 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:57.741584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0d371-834c-4096-895b-0ff62e516ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:57.742040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0d371-834c-4096-895b-0ff62e516ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:57.768843 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-aab0d371-834c-4096-895b-0ff62e516ee6 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:57.770341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-aab0d371-834c-4096-895b-0ff62e516ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:57.802903 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65fc27-a584-494b-8a8e-c7c38d55351b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:57.804703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d65fc27-a584-494b-8a8e-c7c38d55351b None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:57.849814 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:57.878536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 83/157] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:03:57.879900 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0d371-834c-4096-895b-0ff62e516ee6 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:57.997806 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 75/158] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:03:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:03:58.003909 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65fc27-a584-494b-8a8e-c7c38d55351b None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:58.031571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:58.055771 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0d371-834c-4096-895b-0ff62e516ee6 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:58.121602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c9f01-49a5-48f2-a17a-0ef30b22a68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:58.122257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c9f01-49a5-48f2-a17a-0ef30b22a68a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:58.122540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c9f01-49a5-48f2-a17a-0ef30b22a68a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:58.122813 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c9f01-49a5-48f2-a17a-0ef30b22a68a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:58.133987 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c9f01-49a5-48f2-a17a-0ef30b22a68a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:03:58.138923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-513c9f01-49a5-48f2-a17a-0ef30b22a68a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:03:58.138923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:58.138923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:58.138923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:58.138923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:58.138923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:58.138923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:58.138923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:58.138923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:58.138923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:58.138923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:58.138923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:58.138923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:03:58.138923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:03:58.138923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:03:58.138923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:03:58.139872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:03:58.139872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:03:58.139872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:03:58.139872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:03:58.139872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:58.139872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:58.139872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:58.139872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:58.139872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:58.139872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:58.139872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:58.139872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:58.139872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:58.139872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:58.139872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:58.139872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:58.140672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:58.140672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:58.140672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:58.140672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:58.140672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:03:58.140672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:03:58.140672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:03:58.140672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:03:58.140672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:03:58.140672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:03:58.140672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:03:58.140672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:03:58.140672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 84/159] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:03:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:03:58.144994 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c9f01-49a5-48f2-a17a-0ef30b22a68a None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:58.166450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d65fc27-a584-494b-8a8e-c7c38d55351b None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:58.201377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:58.230248 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45971d6f-cb7d-4e15-953d-9cd6c09eb949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:58.230911 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45971d6f-cb7d-4e15-953d-9cd6c09eb949 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:58.231197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45971d6f-cb7d-4e15-953d-9cd6c09eb949 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:58.231467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45971d6f-cb7d-4e15-953d-9cd6c09eb949 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:58.251391 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45971d6f-cb7d-4e15-953d-9cd6c09eb949 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:03:58.256507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-45971d6f-cb7d-4e15-953d-9cd6c09eb949 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:03:58.256507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:58.256507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:58.256507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:58.256507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:58.256507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:58.256507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:58.256507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:58.256507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:58.256507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:58.256507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:58.256507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:58.256507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:03:58.256507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:03:58.256507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:03:58.256507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:03:58.257572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:03:58.257572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:03:58.257572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:03:58.257572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:03:58.257572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:58.257572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:58.257572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:58.257572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:58.257572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:58.257572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:58.257572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:58.257572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:58.257572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:58.257572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:58.257572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:58.257572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:58.258521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:58.258521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:58.258521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:58.258521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:58.258521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:03:58.258521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:03:58.258521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:03:58.258521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:03:58.258521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:03:58.258521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:03:58.258521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:03:58.258521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:03:58.258521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 76/160] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:03:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:03:58.259211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45971d6f-cb7d-4e15-953d-9cd6c09eb949 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:58.286100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c9f01-49a5-48f2-a17a-0ef30b22a68a None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:58.355307 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbd374-9947-48ae-b74b-0c377259c55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:58.355897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbd374-9947-48ae-b74b-0c377259c55c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:58.356177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbd374-9947-48ae-b74b-0c377259c55c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:58.356451 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bbd374-9947-48ae-b74b-0c377259c55c None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:58.357686 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbd374-9947-48ae-b74b-0c377259c55c None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:58.359144 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bbd374-9947-48ae-b74b-0c377259c55c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:58.361034 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 85/161] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:03:58 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:58.361728 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbd374-9947-48ae-b74b-0c377259c55c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:58.392462 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:58.417240 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45971d6f-cb7d-4e15-953d-9cd6c09eb949 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:58.480916 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986cbf0-149c-4d57-a7b8-a8a5eb2a2935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:58.481541 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986cbf0-149c-4d57-a7b8-a8a5eb2a2935 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:58.481819 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986cbf0-149c-4d57-a7b8-a8a5eb2a2935 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:58.482091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986cbf0-149c-4d57-a7b8-a8a5eb2a2935 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:58.482813 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986cbf0-149c-4d57-a7b8-a8a5eb2a2935 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:58.483195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986cbf0-149c-4d57-a7b8-a8a5eb2a2935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:58.484512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986cbf0-149c-4d57-a7b8-a8a5eb2a2935 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:58.510573 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 77/162] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:03:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:58.511437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986cbf0-149c-4d57-a7b8-a8a5eb2a2935 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:58.541428 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bbd374-9947-48ae-b74b-0c377259c55c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:58.561263 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:58.585533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861cb28-0687-44ea-8169-39becd840917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:58.586521 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861cb28-0687-44ea-8169-39becd840917 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:58.587017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861cb28-0687-44ea-8169-39becd840917 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:58.587480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6861cb28-0687-44ea-8169-39becd840917 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:58.588385 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6861cb28-0687-44ea-8169-39becd840917 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:58.588910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6861cb28-0687-44ea-8169-39becd840917 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:58.590480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6861cb28-0687-44ea-8169-39becd840917 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:58.600179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 86/163] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:03:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:58.601440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861cb28-0687-44ea-8169-39becd840917 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:58.707672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6861cb28-0687-44ea-8169-39becd840917 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:58.742020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986cbf0-149c-4d57-a7b8-a8a5eb2a2935 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:58.766101 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62356071-b8ef-4eb6-8d03-dfedab1dcd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:58.767463 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62356071-b8ef-4eb6-8d03-dfedab1dcd18 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:58.767959 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62356071-b8ef-4eb6-8d03-dfedab1dcd18 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:58.768425 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62356071-b8ef-4eb6-8d03-dfedab1dcd18 None admin] PATH_INFO: `/v3/users/1cdae040f69d4994bc484e82a80c2139` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:58.773574 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62356071-b8ef-4eb6-8d03-dfedab1dcd18 None admin] RBAC: Authorizing `identity:get_user(user_id=1cdae040f69d4994bc484e82a80c2139)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:58.775415 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62356071-b8ef-4eb6-8d03-dfedab1dcd18 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:58.777660 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 87/164] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:03:58 2022] GET /identity/v3/users/1cdae040f69d4994bc484e82a80c2139 => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:58.778712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62356071-b8ef-4eb6-8d03-dfedab1dcd18 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:58.826416 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1e9d8-6b4e-4c27-b7ff-6f13b080c680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:58.827054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1e9d8-6b4e-4c27-b7ff-6f13b080c680 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:58.827333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1e9d8-6b4e-4c27-b7ff-6f13b080c680 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:58.827606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f1e9d8-6b4e-4c27-b7ff-6f13b080c680 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:58.828408 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1e9d8-6b4e-4c27-b7ff-6f13b080c680 None admin] RBAC: Adding query filter params (role.id=8fc75c351a0349e294deeed1ff391c4c, scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef, user.id=e88786550f274e61877748f90084c3f4) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:03:58.828740 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1e9d8-6b4e-4c27-b7ff-6f13b080c680 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:58.830207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f1e9d8-6b4e-4c27-b7ff-6f13b080c680 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:58.839478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:58.841994 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 78/165] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:03:58 2022] GET /identity/v3/role_assignments?user.id=e88786550f274e61877748f90084c3f4&scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef&role.id=8fc75c351a0349e294deeed1ff391c4c => generated 608 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:58.848504 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1e9d8-6b4e-4c27-b7ff-6f13b080c680 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:58.870969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:58.997618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62356071-b8ef-4eb6-8d03-dfedab1dcd18 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:59.073636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f1e9d8-6b4e-4c27-b7ff-6f13b080c680 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:59.077166 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dde62b-6d56-4a23-a62c-fe069f7308dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:59.077909 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dde62b-6d56-4a23-a62c-fe069f7308dd None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:59.078194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dde62b-6d56-4a23-a62c-fe069f7308dd None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:59.078475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dde62b-6d56-4a23-a62c-fe069f7308dd None admin] PATH_INFO: `/v3/users/e88786550f274e61877748f90084c3f4` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:59.084396 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dde62b-6d56-4a23-a62c-fe069f7308dd None admin] RBAC: Authorizing `identity:get_user(user_id=e88786550f274e61877748f90084c3f4)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:59.086025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dde62b-6d56-4a23-a62c-fe069f7308dd None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:59.087918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 88/166] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:03:58 2022] GET /identity/v3/users/e88786550f274e61877748f90084c3f4 => generated 275 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:59.089001 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9237e09-27c4-4e14-b849-6f5b48e26b7f None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:59.089289 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9237e09-27c4-4e14-b849-6f5b48e26b7f None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:59.089569 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9237e09-27c4-4e14-b849-6f5b48e26b7f None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:59.091094 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 89/167] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:03:59.097530 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179803dd-a300-410d-89f4-b69fd197d629 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:59.097825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179803dd-a300-410d-89f4-b69fd197d629 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:59.098092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179803dd-a300-410d-89f4-b69fd197d629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:59.127138 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-179803dd-a300-410d-89f4-b69fd197d629 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:03:59.128399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-179803dd-a300-410d-89f4-b69fd197d629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:03:59.154044 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e524eae-955a-4b6c-9db6-c560a97c3aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:59.154693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e524eae-955a-4b6c-9db6-c560a97c3aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:59.154974 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e524eae-955a-4b6c-9db6-c560a97c3aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:59.155251 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e524eae-955a-4b6c-9db6-c560a97c3aa4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:59.231573 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 90/168] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:03:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:03:59.232808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179803dd-a300-410d-89f4-b69fd197d629 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:59.286234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:59.311267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:59.317789 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e524eae-955a-4b6c-9db6-c560a97c3aa4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:59.321983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e524eae-955a-4b6c-9db6-c560a97c3aa4 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:59.360553 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179803dd-a300-410d-89f4-b69fd197d629 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:59.422239 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432c956-d554-45e9-a489-57c44d03be24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:59.423209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432c956-d554-45e9-a489-57c44d03be24 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:59.423619 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432c956-d554-45e9-a489-57c44d03be24 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:59.424028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432c956-d554-45e9-a489-57c44d03be24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:59.470764 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 79/169] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:03:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:03:59.476928 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e524eae-955a-4b6c-9db6-c560a97c3aa4 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:59.534360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432c956-d554-45e9-a489-57c44d03be24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:59.536799 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432c956-d554-45e9-a489-57c44d03be24 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:59.578258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:59.617784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:59.626858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e524eae-955a-4b6c-9db6-c560a97c3aa4 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:59.654899 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 91/170] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:03:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:03:59.656040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432c956-d554-45e9-a489-57c44d03be24 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:59.701414 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46c1d4-a42a-4214-a79b-6d8d12bd5cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:59.702210 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46c1d4-a42a-4214-a79b-6d8d12bd5cce None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:59.702713 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46c1d4-a42a-4214-a79b-6d8d12bd5cce None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:59.703198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46c1d4-a42a-4214-a79b-6d8d12bd5cce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:59.722006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46c1d4-a42a-4214-a79b-6d8d12bd5cce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:03:59.727748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-9e46c1d4-a42a-4214-a79b-6d8d12bd5cce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:03:59.727748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:03:59.727748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:03:59.727748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:03:59.727748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:03:59.727748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:03:59.727748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:03:59.727748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:03:59.727748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:03:59.727748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:03:59.727748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:03:59.727748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:03:59.727748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:03:59.727748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:03:59.727748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:03:59.727748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:03:59.728632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:03:59.728632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:03:59.728632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:03:59.728632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:03:59.728632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:03:59.728632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:03:59.728632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:03:59.728632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:03:59.728632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:03:59.728632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:03:59.728632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:03:59.728632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:03:59.728632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:03:59.728632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:03:59.728632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:03:59.728632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:03:59.729430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:03:59.729430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:03:59.729430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:03:59.729430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:03:59.729430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:03:59.729430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:03:59.729430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:03:59.729430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:03:59.729430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:03:59.729430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:03:59.729430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:03:59.729430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:03:59.732017 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 80/171] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:03:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:03:59.733128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46c1d4-a42a-4214-a79b-6d8d12bd5cce None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:59.779292 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432c956-d554-45e9-a489-57c44d03be24 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:59.832825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42abc3-74e8-4e5f-a44f-45f10d7bf568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:59.833403 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42abc3-74e8-4e5f-a44f-45f10d7bf568 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:59.833677 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42abc3-74e8-4e5f-a44f-45f10d7bf568 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:59.833942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb42abc3-74e8-4e5f-a44f-45f10d7bf568 None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:59.839712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42abc3-74e8-4e5f-a44f-45f10d7bf568 None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:59.841206 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb42abc3-74e8-4e5f-a44f-45f10d7bf568 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:59.842923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 92/172] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:03:59 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:59.843687 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42abc3-74e8-4e5f-a44f-45f10d7bf568 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:03:59.859232 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:03:59.877979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46c1d4-a42a-4214-a79b-6d8d12bd5cce None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:03:59.947196 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2b198-807d-4dc5-be82-907685e4ecb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:03:59.947848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2b198-807d-4dc5-be82-907685e4ecb1 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:03:59.948123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2b198-807d-4dc5-be82-907685e4ecb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:03:59.948395 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2b198-807d-4dc5-be82-907685e4ecb1 None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:03:59.950622 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2b198-807d-4dc5-be82-907685e4ecb1 None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:03:59.952318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2b198-807d-4dc5-be82-907685e4ecb1 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:03:59.955376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 81/173] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:03:59 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:03:59.956149 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2b198-807d-4dc5-be82-907685e4ecb1 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:00.020552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:00.035517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb42abc3-74e8-4e5f-a44f-45f10d7bf568 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:00.074863 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c672124-8a8f-46d4-9062-0c25448bf414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:00.075848 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c672124-8a8f-46d4-9062-0c25448bf414 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:00.076321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c672124-8a8f-46d4-9062-0c25448bf414 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:00.076835 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c672124-8a8f-46d4-9062-0c25448bf414 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:00.077784 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c672124-8a8f-46d4-9062-0c25448bf414 None admin] RBAC: Adding query filter params (name=member) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:00.078292 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c672124-8a8f-46d4-9062-0c25448bf414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:00.079876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c672124-8a8f-46d4-9062-0c25448bf414 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:00.089358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 93/174] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:03:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:00.090887 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28146970-2427-4708-a444-45743bf7e36b None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:00.091376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28146970-2427-4708-a444-45743bf7e36b None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:00.091836 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28146970-2427-4708-a444-45743bf7e36b None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:00.093285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 94/175] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:00.094239 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28146970-2427-4708-a444-45743bf7e36b None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:00.138536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2b198-807d-4dc5-be82-907685e4ecb1 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:00.191645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d0c40e-2e66-4249-a2ff-c12b2278b8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:00.191645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0c40e-2e66-4249-a2ff-c12b2278b8be None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:00.191645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0c40e-2e66-4249-a2ff-c12b2278b8be None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:00.191645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0c40e-2e66-4249-a2ff-c12b2278b8be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:00.193036 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0c40e-2e66-4249-a2ff-c12b2278b8be None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef, user.id=1cdae040f69d4994bc484e82a80c2139) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:00.193566 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0c40e-2e66-4249-a2ff-c12b2278b8be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:00.195324 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d0c40e-2e66-4249-a2ff-c12b2278b8be None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:00.209558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 82/176] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:03:59 2022] GET /identity/v3/role_assignments?user.id=1cdae040f69d4994bc484e82a80c2139&scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef&role.id=35205394c8af4c52a61b12f92ba16644 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:00.211148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b213cd4-7981-4e45-b4b0-bd813048d8ec None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:00.211704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b213cd4-7981-4e45-b4b0-bd813048d8ec None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:00.212305 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b213cd4-7981-4e45-b4b0-bd813048d8ec None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:00.214007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 83/177] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:00.215166 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b213cd4-7981-4e45-b4b0-bd813048d8ec None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:00.221010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:00.399903 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28146970-2427-4708-a444-45743bf7e36b None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:00.427518 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:00.446976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b213cd4-7981-4e45-b4b0-bd813048d8ec None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:00.462854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec7468-11aa-4624-a821-07f2bf2d9134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:00.463757 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec7468-11aa-4624-a821-07f2bf2d9134 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:00.464107 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec7468-11aa-4624-a821-07f2bf2d9134 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:00.464450 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec7468-11aa-4624-a821-07f2bf2d9134 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:00.465404 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec7468-11aa-4624-a821-07f2bf2d9134 None admin] RBAC: Adding query filter params (role.id=80e5528d7c8742f78e29e64c1216176c, scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef, user.id=e88786550f274e61877748f90084c3f4) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:00.467875 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec7468-11aa-4624-a821-07f2bf2d9134 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:00.470449 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec7468-11aa-4624-a821-07f2bf2d9134 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:00.478105 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 95/178] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:00 2022] GET /identity/v3/role_assignments?user.id=e88786550f274e61877748f90084c3f4&scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef&role.id=80e5528d7c8742f78e29e64c1216176c => generated 608 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:00.479351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b1ef1-bbc3-4700-a36e-da7b559f39bd None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:00.479635 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b1ef1-bbc3-4700-a36e-da7b559f39bd None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:00.479907 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b1ef1-bbc3-4700-a36e-da7b559f39bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:00.507150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5a40b-4cc4-4ba9-a3ab-45cdf48eb9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:00.508897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-ec9b1ef1-bbc3-4700-a36e-da7b559f39bd None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:00.509258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5a40b-4cc4-4ba9-a3ab-45cdf48eb9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:00.509886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5a40b-4cc4-4ba9-a3ab-45cdf48eb9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:00.510515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5a40b-4cc4-4ba9-a3ab-45cdf48eb9e5 None admin] PATH_INFO: `/v3/users/e88786550f274e61877748f90084c3f4` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:00.511022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-ec9b1ef1-bbc3-4700-a36e-da7b559f39bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:00.512417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5a40b-4cc4-4ba9-a3ab-45cdf48eb9e5 None admin] RBAC: Authorizing `identity:get_user(user_id=e88786550f274e61877748f90084c3f4)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:00.514158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5a40b-4cc4-4ba9-a3ab-45cdf48eb9e5 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:00.516908 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 84/179] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:00 2022] GET /identity/v3/users/e88786550f274e61877748f90084c3f4 => generated 275 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:00.518140 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c6676-a4ad-4c76-9244-269e6a872f9d None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:00.518427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c6676-a4ad-4c76-9244-269e6a872f9d None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:00.518701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c6676-a4ad-4c76-9244-269e6a872f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:00.543627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-d29c6676-a4ad-4c76-9244-269e6a872f9d None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:00.545330 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d29c6676-a4ad-4c76-9244-269e6a872f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:00.608452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 96/180] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:00.618817 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea469df-6ba0-4a95-9b8e-fa400c995c3f None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:00.619113 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea469df-6ba0-4a95-9b8e-fa400c995c3f None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:00.619669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea469df-6ba0-4a95-9b8e-fa400c995c3f None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:00.621401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 97/181] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:00.623590 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9885c3-4485-43c7-9da7-3c6c1ca0231c None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:00.623871 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9885c3-4485-43c7-9da7-3c6c1ca0231c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:00.624147 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9885c3-4485-43c7-9da7-3c6c1ca0231c None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:00.626971 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 98/182] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:00.628004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9885c3-4485-43c7-9da7-3c6c1ca0231c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:00.638434 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 85/183] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:00.639705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56584907-67fb-4292-94fd-967afdedcf13 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:00.639975 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56584907-67fb-4292-94fd-967afdedcf13 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:00.640325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56584907-67fb-4292-94fd-967afdedcf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:00.662637 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-56584907-67fb-4292-94fd-967afdedcf13 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:00.664106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-56584907-67fb-4292-94fd-967afdedcf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:00.713463 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:00.741169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 86/184] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:00.752868 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1b855-1c55-4ba1-aa82-35453d3908eb None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:00.752868 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1b855-1c55-4ba1-aa82-35453d3908eb None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:00.752868 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d1b855-1c55-4ba1-aa82-35453d3908eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:00.759529 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9885c3-4485-43c7-9da7-3c6c1ca0231c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:00.761069 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-c7d1b855-1c55-4ba1-aa82-35453d3908eb None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:00.762360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-c7d1b855-1c55-4ba1-aa82-35453d3908eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:00.776720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:00.794242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414d854-3844-431a-a1e8-c8166ba90c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:00.794948 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414d854-3844-431a-a1e8-c8166ba90c83 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:00.795138 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414d854-3844-431a-a1e8-c8166ba90c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:00.795413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414d854-3844-431a-a1e8-c8166ba90c83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:00.832339 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 87/185] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:00.833223 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1b855-1c55-4ba1-aa82-35453d3908eb None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:00.869921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414d854-3844-431a-a1e8-c8166ba90c83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:00.871379 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414d854-3844-431a-a1e8-c8166ba90c83 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:00.930966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:00.931721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d1b855-1c55-4ba1-aa82-35453d3908eb None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:00.965956 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06aebf0-dd55-460d-8dcc-65cc733c6de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:00.966634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06aebf0-dd55-460d-8dcc-65cc733c6de2 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:00.966917 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06aebf0-dd55-460d-8dcc-65cc733c6de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:00.967244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06aebf0-dd55-460d-8dcc-65cc733c6de2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:00.970627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 99/186] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:00.971468 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414d854-3844-431a-a1e8-c8166ba90c83 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:00.980454 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06aebf0-dd55-460d-8dcc-65cc733c6de2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:00.985427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-a06aebf0-dd55-460d-8dcc-65cc733c6de2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:04:00.985427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:00.985427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:00.985427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:00.985427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:00.985427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:00.985427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:00.985427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:00.985427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:00.985427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:00.985427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:00.985427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:00.985427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:00.985427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:00.985427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:00.985427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:00.986336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:00.986336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:00.986336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:00.986336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:00.986336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:00.986336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:00.986336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:00.986336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:00.986336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:00.986336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:00.986336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:00.986336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:00.986336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:00.986336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:00.986336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:00.986336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:00.987122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:00.987122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:00.987122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:00.987122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:00.987122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:00.987122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:00.987122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:00.987122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:00.987122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:00.987122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:00.987122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:04:00.987122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:00.987122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 88/187] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:04:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:00.989418 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06aebf0-dd55-460d-8dcc-65cc733c6de2 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:01.001708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:01.073042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414d854-3844-431a-a1e8-c8166ba90c83 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:01.092132 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06aebf0-dd55-460d-8dcc-65cc733c6de2 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:01.108361 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdf9e4-506e-4cfc-a9d3-fb90bdb79ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:01.109049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdf9e4-506e-4cfc-a9d3-fb90bdb79ace None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:01.109329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdf9e4-506e-4cfc-a9d3-fb90bdb79ace None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:01.109613 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdf9e4-506e-4cfc-a9d3-fb90bdb79ace None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:01.119468 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebdf9e4-506e-4cfc-a9d3-fb90bdb79ace None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:01.124387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-7ebdf9e4-506e-4cfc-a9d3-fb90bdb79ace None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:01.124387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:01.124387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:01.124387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:01.124387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:01.124387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:01.124387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:01.124387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:01.124387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:01.124387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:01.124387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:01.124387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:01.124387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:01.124387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:01.124387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:01.124387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:01.125426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:01.125426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:01.125426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:01.125426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:01.125426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:01.125426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:01.125426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:01.125426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:01.125426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:01.125426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:01.125426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:01.125426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:01.125426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:01.125426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:01.125426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:01.125426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:01.126327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:01.126327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:01.126327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:01.126327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:01.126327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:01.126327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:01.126327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:01.126327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:01.126327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:01.126327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:01.126327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:01.126327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:01.126327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 100/188] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:04:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:01.128393 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907343de-f145-4052-9666-047538e1c567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:01.129067 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907343de-f145-4052-9666-047538e1c567 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:01.129349 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907343de-f145-4052-9666-047538e1c567 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:01.129623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907343de-f145-4052-9666-047538e1c567 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:01.130010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdf9e4-506e-4cfc-a9d3-fb90bdb79ace None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:01.130844 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907343de-f145-4052-9666-047538e1c567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:01.132299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907343de-f145-4052-9666-047538e1c567 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:01.133928 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 89/189] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:01.137537 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907343de-f145-4052-9666-047538e1c567 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:01.156449 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:01.214663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:01.233962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdf9e4-506e-4cfc-a9d3-fb90bdb79ace None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:01.236848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907343de-f145-4052-9666-047538e1c567 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:01.267514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc45e71-8ab2-4b80-9b79-d921df1bffe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:01.268150 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc45e71-8ab2-4b80-9b79-d921df1bffe5 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:01.268433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc45e71-8ab2-4b80-9b79-d921df1bffe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:01.268739 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc45e71-8ab2-4b80-9b79-d921df1bffe5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:01.270426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e56b88-77e4-4644-8f7a-099019e546c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:01.271065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e56b88-77e4-4644-8f7a-099019e546c8 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:01.271343 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e56b88-77e4-4644-8f7a-099019e546c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:01.271615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e56b88-77e4-4644-8f7a-099019e546c8 None admin] PATH_INFO: `/v3/projects/c60d517a79da4691a934dbee19a2ad95` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:01.272802 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e56b88-77e4-4644-8f7a-099019e546c8 None admin] RBAC: Authorizing `identity:get_project(project_id=c60d517a79da4691a934dbee19a2ad95)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:01.274275 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e56b88-77e4-4644-8f7a-099019e546c8 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:01.276038 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 90/190] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:01 2022] GET /identity/v3/projects/c60d517a79da4691a934dbee19a2ad95 => generated 312 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:01.276808 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e56b88-77e4-4644-8f7a-099019e546c8 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:01.280254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc45e71-8ab2-4b80-9b79-d921df1bffe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:01.285760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-2bc45e71-8ab2-4b80-9b79-d921df1bffe5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:04:01.285760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:01.285760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:01.285760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:01.285760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:01.285760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:01.285760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:01.285760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:01.285760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:01.285760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:01.285760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:01.285760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:01.285760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:01.285760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:01.285760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:01.285760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:01.286698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:01.286698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:01.286698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:01.286698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:01.286698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:01.286698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:01.286698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:01.286698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:01.286698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:01.286698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:01.286698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:01.286698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:01.286698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:01.286698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:01.286698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:01.286698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:01.292335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:01.292335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:01.292335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:01.292335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:01.292335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:01.292335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:01.292335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:01.292335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:01.292335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:01.292335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:01.292335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:04:01.292335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:01.292335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 101/191] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:01.292335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc45e71-8ab2-4b80-9b79-d921df1bffe5 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:01.371794 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:01.377151 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e56b88-77e4-4644-8f7a-099019e546c8 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:01.390465 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc45e71-8ab2-4b80-9b79-d921df1bffe5 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:01.412536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de10c9-770e-4359-b86a-77b1767d6678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:01.413173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de10c9-770e-4359-b86a-77b1767d6678 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:01.413450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de10c9-770e-4359-b86a-77b1767d6678 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:01.413724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de10c9-770e-4359-b86a-77b1767d6678 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:01.414433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de10c9-770e-4359-b86a-77b1767d6678 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:01.414734 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de10c9-770e-4359-b86a-77b1767d6678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:01.416121 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de10c9-770e-4359-b86a-77b1767d6678 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:01.423648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7fde2-8b27-40ff-a412-1e041ee6805f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:01.424092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 91/192] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:01.424255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7fde2-8b27-40ff-a412-1e041ee6805f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:01.424530 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7fde2-8b27-40ff-a412-1e041ee6805f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:01.424824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7fde2-8b27-40ff-a412-1e041ee6805f None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:01.424912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de10c9-770e-4359-b86a-77b1767d6678 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:01.425592 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7fde2-8b27-40ff-a412-1e041ee6805f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:01.425895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7fde2-8b27-40ff-a412-1e041ee6805f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:01.427267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7fde2-8b27-40ff-a412-1e041ee6805f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:01.435497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 102/193] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:01.436283 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7fde2-8b27-40ff-a412-1e041ee6805f None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:01.438413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:01.528042 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de10c9-770e-4359-b86a-77b1767d6678 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:01.536222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7fde2-8b27-40ff-a412-1e041ee6805f None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:01.559951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea75da-c040-4bf6-9e19-435dd5e58599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:01.560845 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea75da-c040-4bf6-9e19-435dd5e58599 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:01.561126 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea75da-c040-4bf6-9e19-435dd5e58599 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:01.561406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea75da-c040-4bf6-9e19-435dd5e58599 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:01.562074 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea75da-c040-4bf6-9e19-435dd5e58599 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:01.563505 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea75da-c040-4bf6-9e19-435dd5e58599 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:01.569373 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacaac3-cc50-49ff-aedf-7f424503db7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:01.569987 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacaac3-cc50-49ff-aedf-7f424503db7c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:01.570272 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacaac3-cc50-49ff-aedf-7f424503db7c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:01.570555 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacaac3-cc50-49ff-aedf-7f424503db7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:01.571375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacaac3-cc50-49ff-aedf-7f424503db7c None admin] RBAC: Adding query filter params (role.id=8fc75c351a0349e294deeed1ff391c4c, scope.project.id=c60d517a79da4691a934dbee19a2ad95, user.id=e88786550f274e61877748f90084c3f4) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:01.571899 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacaac3-cc50-49ff-aedf-7f424503db7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:01.574333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacaac3-cc50-49ff-aedf-7f424503db7c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:01.581073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 103/194] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:01 2022] GET /identity/v3/role_assignments?user.id=e88786550f274e61877748f90084c3f4&scope.project.id=c60d517a79da4691a934dbee19a2ad95&role.id=8fc75c351a0349e294deeed1ff391c4c => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:01.582443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacaac3-cc50-49ff-aedf-7f424503db7c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:01.592209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 92/195] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:04:01 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:04:01.593164 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea75da-c040-4bf6-9e19-435dd5e58599 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:01.604767 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:01.703553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:01.752347 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacaac3-cc50-49ff-aedf-7f424503db7c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:01.754101 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea75da-c040-4bf6-9e19-435dd5e58599 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:01.796508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b30309-93bc-44c9-b20e-428c4efb1e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:01.797955 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b30309-93bc-44c9-b20e-428c4efb1e02 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:01.797955 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b30309-93bc-44c9-b20e-428c4efb1e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:01.797955 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b30309-93bc-44c9-b20e-428c4efb1e02 None admin] PATH_INFO: `/v3/users/e1ee92ad9ee445fdb5e930f83bc05d9e` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:01.802618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687dcad7-001b-4986-b36e-348c38b50176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:01.803248 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687dcad7-001b-4986-b36e-348c38b50176 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:01.803551 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687dcad7-001b-4986-b36e-348c38b50176 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:01.803802 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687dcad7-001b-4986-b36e-348c38b50176 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:01.804601 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687dcad7-001b-4986-b36e-348c38b50176 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:01.805026 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687dcad7-001b-4986-b36e-348c38b50176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:01.806699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687dcad7-001b-4986-b36e-348c38b50176 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:01.817141 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b30309-93bc-44c9-b20e-428c4efb1e02 None admin] RBAC: Authorizing `identity:get_user(user_id=e1ee92ad9ee445fdb5e930f83bc05d9e)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:01.819831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b30309-93bc-44c9-b20e-428c4efb1e02 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:01.822539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 93/196] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:01 2022] GET /identity/v3/users/e1ee92ad9ee445fdb5e930f83bc05d9e => generated 283 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:01.823346 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b30309-93bc-44c9-b20e-428c4efb1e02 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:01.823873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 104/197] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:01.829476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687dcad7-001b-4986-b36e-348c38b50176 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:01.935881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:01.955380 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b30309-93bc-44c9-b20e-428c4efb1e02 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:01.977946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687dcad7-001b-4986-b36e-348c38b50176 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:01.986630 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:02.005281 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1013f9-a909-4f69-bb67-4774b866f5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:02.005690 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1013f9-a909-4f69-bb67-4774b866f5fd None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:02.005970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1013f9-a909-4f69-bb67-4774b866f5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:02.006243 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1013f9-a909-4f69-bb67-4774b866f5fd None admin] PATH_INFO: `/v3/users/e1ee92ad9ee445fdb5e930f83bc05d9e` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:02.007475 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1013f9-a909-4f69-bb67-4774b866f5fd None admin] RBAC: Authorizing `identity:get_user(user_id=e1ee92ad9ee445fdb5e930f83bc05d9e)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:02.008959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1013f9-a909-4f69-bb67-4774b866f5fd None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:02.010574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 94/198] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:01 2022] GET /identity/v3/users/e1ee92ad9ee445fdb5e930f83bc05d9e => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:02.011335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1013f9-a909-4f69-bb67-4774b866f5fd None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:02.024347 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa74ea-361e-45d2-9f22-f04ca1359da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:02.025334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa74ea-361e-45d2-9f22-f04ca1359da6 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:02.025673 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa74ea-361e-45d2-9f22-f04ca1359da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:02.025960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa74ea-361e-45d2-9f22-f04ca1359da6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:02.126030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa74ea-361e-45d2-9f22-f04ca1359da6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:02.127625 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa74ea-361e-45d2-9f22-f04ca1359da6 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:02.127843 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1013f9-a909-4f69-bb67-4774b866f5fd None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:02.168849 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d469e-05e8-40e2-92e6-dca7207a631f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:02.169473 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d469e-05e8-40e2-92e6-dca7207a631f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:02.169821 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d469e-05e8-40e2-92e6-dca7207a631f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:02.170081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d469e-05e8-40e2-92e6-dca7207a631f None admin] PATH_INFO: `/v3/users/e38c92e5198e45c3bb8ef525b2fec7b9` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:02.171160 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d469e-05e8-40e2-92e6-dca7207a631f None admin] RBAC: Authorizing `identity:get_user(user_id=e38c92e5198e45c3bb8ef525b2fec7b9)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:02.172602 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d469e-05e8-40e2-92e6-dca7207a631f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:02.174443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 95/199] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:02 2022] GET /identity/v3/users/e38c92e5198e45c3bb8ef525b2fec7b9 => generated 289 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:02.174976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d469e-05e8-40e2-92e6-dca7207a631f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:02.238364 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:02.241777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:02.286622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 105/200] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:02.286622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa74ea-361e-45d2-9f22-f04ca1359da6 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:02.298929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d469e-05e8-40e2-92e6-dca7207a631f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:02.345954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b1b47-bf59-41c3-9bfe-8d54f78f1cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:02.347024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b1b47-bf59-41c3-9bfe-8d54f78f1cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:02.347922 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b1b47-bf59-41c3-9bfe-8d54f78f1cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:02.348402 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b1b47-bf59-41c3-9bfe-8d54f78f1cb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:02.410543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa74ea-361e-45d2-9f22-f04ca1359da6 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:02.433930 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b1b47-bf59-41c3-9bfe-8d54f78f1cb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:02.435669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b1b47-bf59-41c3-9bfe-8d54f78f1cb2 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:02.454185 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc4faa-4d8f-45e8-b767-0a04d3bcab95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:02.455200 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc4faa-4d8f-45e8-b767-0a04d3bcab95 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:02.455693 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc4faa-4d8f-45e8-b767-0a04d3bcab95 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:02.456246 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffc4faa-4d8f-45e8-b767-0a04d3bcab95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:02.498704 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:02.499568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:02.529705 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc4faa-4d8f-45e8-b767-0a04d3bcab95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:02.531536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffc4faa-4d8f-45e8-b767-0a04d3bcab95 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:02.545849 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 96/201] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:02.546718 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b1b47-bf59-41c3-9bfe-8d54f78f1cb2 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:02.636273 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 106/202] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:02.637650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc4faa-4d8f-45e8-b767-0a04d3bcab95 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:02.663312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b1b47-bf59-41c3-9bfe-8d54f78f1cb2 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:02.699620 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c551668-9581-4946-9c86-7803a65f4d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:02.700361 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c551668-9581-4946-9c86-7803a65f4d11 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:02.700667 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c551668-9581-4946-9c86-7803a65f4d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:02.701053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c551668-9581-4946-9c86-7803a65f4d11 None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:02.702224 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c551668-9581-4946-9c86-7803a65f4d11 None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:02.703762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c551668-9581-4946-9c86-7803a65f4d11 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:02.711002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 97/203] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:02 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:02.711002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c551668-9581-4946-9c86-7803a65f4d11 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:02.730120 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:02.748892 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:02.770517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffc4faa-4d8f-45e8-b767-0a04d3bcab95 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:02.814516 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35e6dd-245b-422c-9d00-7d592eb313e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:02.815167 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35e6dd-245b-422c-9d00-7d592eb313e6 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:02.815429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35e6dd-245b-422c-9d00-7d592eb313e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:02.815777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35e6dd-245b-422c-9d00-7d592eb313e6 None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:02.817472 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed35e6dd-245b-422c-9d00-7d592eb313e6 None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:02.819330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed35e6dd-245b-422c-9d00-7d592eb313e6 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:02.821870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 107/204] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:02 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:02.822455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35e6dd-245b-422c-9d00-7d592eb313e6 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:02.850149 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c551668-9581-4946-9c86-7803a65f4d11 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:02.903243 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be7bc9-e24e-4365-9787-d7758bf0abfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:02.904267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be7bc9-e24e-4365-9787-d7758bf0abfb None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:02.904796 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be7bc9-e24e-4365-9787-d7758bf0abfb None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:02.905503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be7bc9-e24e-4365-9787-d7758bf0abfb None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:02.907833 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be7bc9-e24e-4365-9787-d7758bf0abfb None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:02.911662 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be7bc9-e24e-4365-9787-d7758bf0abfb None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:02.914014 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 98/205] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:02 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:02.915200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be7bc9-e24e-4365-9787-d7758bf0abfb None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:02.950573 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35e6dd-245b-422c-9d00-7d592eb313e6 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:02.986087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a425e-e8dd-4b18-ac7f-f4d970366511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:02.986795 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a425e-e8dd-4b18-ac7f-f4d970366511 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:02.986988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a425e-e8dd-4b18-ac7f-f4d970366511 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:02.987254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a425e-e8dd-4b18-ac7f-f4d970366511 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:02.988666 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a425e-e8dd-4b18-ac7f-f4d970366511 None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92, user.id=e1ee92ad9ee445fdb5e930f83bc05d9e) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:02.989209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a425e-e8dd-4b18-ac7f-f4d970366511 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:02.990661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1a425e-e8dd-4b18-ac7f-f4d970366511 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:03.004653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 108/206] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:02 2022] GET /identity/v3/role_assignments?user.id=e1ee92ad9ee445fdb5e930f83bc05d9e&scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92&role.id=35205394c8af4c52a61b12f92ba16644 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:03.004653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a425e-e8dd-4b18-ac7f-f4d970366511 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:03.010564 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:03.074001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:03.091074 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be7bc9-e24e-4365-9787-d7758bf0abfb None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:03.169295 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f814e-c6f2-44c4-a6d7-e4afef094197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:03.170282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f814e-c6f2-44c4-a6d7-e4afef094197 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:03.170773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f814e-c6f2-44c4-a6d7-e4afef094197 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:03.171239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f814e-c6f2-44c4-a6d7-e4afef094197 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:03.172237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f814e-c6f2-44c4-a6d7-e4afef094197 None admin] RBAC: Adding query filter params (role.id=80e5528d7c8742f78e29e64c1216176c, scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92, user.id=e1ee92ad9ee445fdb5e930f83bc05d9e) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:03.174525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f814e-c6f2-44c4-a6d7-e4afef094197 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:03.176866 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f814e-c6f2-44c4-a6d7-e4afef094197 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:03.185576 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 99/207] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:02 2022] GET /identity/v3/role_assignments?user.id=e1ee92ad9ee445fdb5e930f83bc05d9e&scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92&role.id=80e5528d7c8742f78e29e64c1216176c => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:03.212543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a425e-e8dd-4b18-ac7f-f4d970366511 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:03.252208 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589004f2-5185-4c71-8ea6-654890f4e1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:03.253418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589004f2-5185-4c71-8ea6-654890f4e1eb None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:03.253945 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589004f2-5185-4c71-8ea6-654890f4e1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:03.254417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589004f2-5185-4c71-8ea6-654890f4e1eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:03.255421 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589004f2-5185-4c71-8ea6-654890f4e1eb None admin] RBAC: Adding query filter params (role.id=9e8771d0146c45e6afaf7915e215ac91, scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef, user.id=e38c92e5198e45c3bb8ef525b2fec7b9) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:03.255939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589004f2-5185-4c71-8ea6-654890f4e1eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:03.258032 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589004f2-5185-4c71-8ea6-654890f4e1eb None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:03.265647 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 109/208] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:02 2022] GET /identity/v3/role_assignments?user.id=e38c92e5198e45c3bb8ef525b2fec7b9&scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef&role.id=9e8771d0146c45e6afaf7915e215ac91 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:03.860655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4792593-adca-48a3-8f08-5099f3ea7d6a None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:03.861139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4792593-adca-48a3-8f08-5099f3ea7d6a None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:03.861240 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4792593-adca-48a3-8f08-5099f3ea7d6a None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:03.863277 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 100/209] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:03.869827 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a495342-0bf2-47e0-9e30-49d113daa7b6 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:03.870680 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a495342-0bf2-47e0-9e30-49d113daa7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:03.871158 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a495342-0bf2-47e0-9e30-49d113daa7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:03.896823 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-8a495342-0bf2-47e0-9e30-49d113daa7b6 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:03.898538 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8a495342-0bf2-47e0-9e30-49d113daa7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:03.984019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:03.992945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9cb20-bcbc-4cac-a850-3e6dd62dba03 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:03.993613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9cb20-bcbc-4cac-a850-3e6dd62dba03 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:03.993881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 110/210] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:03.994327 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9cb20-bcbc-4cac-a850-3e6dd62dba03 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:03.995915 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 101/211] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:04:04.006796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c176bfd-ef0b-4002-bc09-62c214d5ba6f None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:04.007087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c176bfd-ef0b-4002-bc09-62c214d5ba6f None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:04.007370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c176bfd-ef0b-4002-bc09-62c214d5ba6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:04.025421 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-5c176bfd-ef0b-4002-bc09-62c214d5ba6f None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:04.025594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9cb20-bcbc-4cac-a850-3e6dd62dba03 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:04.026625 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-5c176bfd-ef0b-4002-bc09-62c214d5ba6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:04.105252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 111/212] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:04.129125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c176bfd-ef0b-4002-bc09-62c214d5ba6f None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:04.141356 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9cb20-bcbc-4cac-a850-3e6dd62dba03 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:04.178782 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4342f2-c5f9-44f4-ad8c-969c8e52645b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:04.179784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4342f2-c5f9-44f4-ad8c-969c8e52645b None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:04.180265 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4342f2-c5f9-44f4-ad8c-969c8e52645b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:04.180771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4342f2-c5f9-44f4-ad8c-969c8e52645b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:04.192480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4342f2-c5f9-44f4-ad8c-969c8e52645b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:04.198438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-8e4342f2-c5f9-44f4-ad8c-969c8e52645b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:04.198438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:04.198438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:04.198438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:04.198438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:04.198438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:04.198438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:04.198438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:04.198438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:04.198438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:04.198438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:04.198438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:04.198438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:04.198438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:04.198438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:04.198438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:04.199335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:04.199335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:04.199335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:04.199335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:04.199335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:04.199335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:04.199335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:04.199335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:04.199335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:04.199335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:04.199335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:04.199335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:04.199335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:04.199335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:04.199335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:04.199335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:04.202058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:04.202058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:04.202058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:04.202058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:04.202058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:04.202058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:04.202058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:04.202058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:04.202058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:04.202058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:04.202058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:04.202058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:04.205803 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 102/213] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:04.211840 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4342f2-c5f9-44f4-ad8c-969c8e52645b None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:04.234913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:04.245263 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c176bfd-ef0b-4002-bc09-62c214d5ba6f None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:04.283333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:04.293921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63cade-6873-47b7-9ae1-9bf6e81f0d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:04.294646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63cade-6873-47b7-9ae1-9bf6e81f0d57 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:04.295011 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63cade-6873-47b7-9ae1-9bf6e81f0d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:04.295382 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63cade-6873-47b7-9ae1-9bf6e81f0d57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:04.296704 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63cade-6873-47b7-9ae1-9bf6e81f0d57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:04.298259 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63cade-6873-47b7-9ae1-9bf6e81f0d57 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:04.300846 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 112/214] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:04.305788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63cade-6873-47b7-9ae1-9bf6e81f0d57 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:04.338204 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4342f2-c5f9-44f4-ad8c-969c8e52645b None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:04.380374 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9e17e-30e9-49c5-a23f-b78bb1418d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:04.381384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9e17e-30e9-49c5-a23f-b78bb1418d61 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:04.381867 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9e17e-30e9-49c5-a23f-b78bb1418d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:04.382427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9e17e-30e9-49c5-a23f-b78bb1418d61 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:04.383471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9e17e-30e9-49c5-a23f-b78bb1418d61 None admin] RBAC: Adding query filter params (name=member) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:04.384044 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9e17e-30e9-49c5-a23f-b78bb1418d61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:04.385895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9e17e-30e9-49c5-a23f-b78bb1418d61 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:04.395355 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 103/215] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:04.402269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9e17e-30e9-49c5-a23f-b78bb1418d61 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:04.443368 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63cade-6873-47b7-9ae1-9bf6e81f0d57 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:04.490110 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07bd1a5-eeed-453f-8745-728fceed40b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:04.490657 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07bd1a5-eeed-453f-8745-728fceed40b5 None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:04.490939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07bd1a5-eeed-453f-8745-728fceed40b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:04.491214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07bd1a5-eeed-453f-8745-728fceed40b5 None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:04.492045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:04.492898 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07bd1a5-eeed-453f-8745-728fceed40b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:04.494797 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07bd1a5-eeed-453f-8745-728fceed40b5 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:04.518460 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9e17e-30e9-49c5-a23f-b78bb1418d61 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:04.534163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 113/216] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:04:04 2022] POST /identity/v3/users => generated 297 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:04:04.575423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f20fcf-c119-47aa-81ec-99110758fb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:04.576383 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f20fcf-c119-47aa-81ec-99110758fb43 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:04.576846 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f20fcf-c119-47aa-81ec-99110758fb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:04.577275 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f20fcf-c119-47aa-81ec-99110758fb43 None admin] PATH_INFO: `/v3/users/6ed69a73796d4f67b1a7458a294afa70` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:04.595937 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f20fcf-c119-47aa-81ec-99110758fb43 None admin] RBAC: Authorizing `identity:get_user(user_id=6ed69a73796d4f67b1a7458a294afa70)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:04.598294 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f20fcf-c119-47aa-81ec-99110758fb43 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:04.600236 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 104/217] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:04 2022] GET /identity/v3/users/6ed69a73796d4f67b1a7458a294afa70 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:04.606911 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07bd1a5-eeed-453f-8745-728fceed40b5 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:04.676070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:04.740266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07bd1a5-eeed-453f-8745-728fceed40b5 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:04.775900 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720791d0-be02-4eec-b477-46faa6aad3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:04.776519 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720791d0-be02-4eec-b477-46faa6aad3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:04.776839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720791d0-be02-4eec-b477-46faa6aad3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:04.777113 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720791d0-be02-4eec-b477-46faa6aad3a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:04.850019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720791d0-be02-4eec-b477-46faa6aad3a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:04.851500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720791d0-be02-4eec-b477-46faa6aad3a9 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:04.906101 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:04.965624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 114/218] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:04.971156 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f20fcf-c119-47aa-81ec-99110758fb43 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:05.080450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f20fcf-c119-47aa-81ec-99110758fb43 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:05.117203 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f0c870-f985-4ed6-9cad-c24ed411f758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:05.117849 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0c870-f985-4ed6-9cad-c24ed411f758 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:05.118127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0c870-f985-4ed6-9cad-c24ed411f758 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:05.118401 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f0c870-f985-4ed6-9cad-c24ed411f758 None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:05.119574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0c870-f985-4ed6-9cad-c24ed411f758 None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:05.121112 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f0c870-f985-4ed6-9cad-c24ed411f758 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:05.123075 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 105/219] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:04 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:05.129751 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720791d0-be02-4eec-b477-46faa6aad3a9 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:05.287584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720791d0-be02-4eec-b477-46faa6aad3a9 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:05.334852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ec764-cfd8-432f-b27a-be23c88c90d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:05.335406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ec764-cfd8-432f-b27a-be23c88c90d4 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:05.335687 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ec764-cfd8-432f-b27a-be23c88c90d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:05.335968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ec764-cfd8-432f-b27a-be23c88c90d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:05.336794 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ec764-cfd8-432f-b27a-be23c88c90d4 None admin] RBAC: Adding query filter params (role.id=8fc75c351a0349e294deeed1ff391c4c, scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92, user.id=6ed69a73796d4f67b1a7458a294afa70) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:05.337106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ec764-cfd8-432f-b27a-be23c88c90d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:05.338554 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ec764-cfd8-432f-b27a-be23c88c90d4 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:05.348334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 115/220] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:05 2022] GET /identity/v3/role_assignments?user.id=6ed69a73796d4f67b1a7458a294afa70&scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92&role.id=8fc75c351a0349e294deeed1ff391c4c => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:05.401871 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b41ef6-e6ab-4ddf-a3d6-a3d41026a20b None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:05.402183 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b41ef6-e6ab-4ddf-a3d6-a3d41026a20b None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:05.402426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b41ef6-e6ab-4ddf-a3d6-a3d41026a20b None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:05.403782 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 106/221] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:05.415100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab3ee3-4dd8-45a2-a023-da9e165373f5 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:05.415406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab3ee3-4dd8-45a2-a023-da9e165373f5 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:05.415649 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab3ee3-4dd8-45a2-a023-da9e165373f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:05.445588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-e7ab3ee3-4dd8-45a2-a023-da9e165373f5 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:05.446827 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e7ab3ee3-4dd8-45a2-a023-da9e165373f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:05.458858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:05.489213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c0c67-90f3-4929-9f9d-700848bc5eaf None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:05.489532 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c0c67-90f3-4929-9f9d-700848bc5eaf None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:05.489761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c0c67-90f3-4929-9f9d-700848bc5eaf None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:05.491153 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 107/222] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:05.498798 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd3014-aa51-45fc-8fab-87f5b44e2b81 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:05.499390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd3014-aa51-45fc-8fab-87f5b44e2b81 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:05.499678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd3014-aa51-45fc-8fab-87f5b44e2b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:05.528993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:05.541347 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-69dd3014-aa51-45fc-8fab-87f5b44e2b81 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:05.543445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-69dd3014-aa51-45fc-8fab-87f5b44e2b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:05.569559 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 116/223] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:05.631723 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 108/224] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:05.635143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab3ee3-4dd8-45a2-a023-da9e165373f5 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:05.663731 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd3014-aa51-45fc-8fab-87f5b44e2b81 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:05.796119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd3014-aa51-45fc-8fab-87f5b44e2b81 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:05.838634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f9c30-b555-4cfe-9438-099727094695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:05.839279 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f9c30-b555-4cfe-9438-099727094695 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:05.839556 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f9c30-b555-4cfe-9438-099727094695 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:05.839826 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2f9c30-b555-4cfe-9438-099727094695 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:05.848743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:05.855817 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab3ee3-4dd8-45a2-a023-da9e165373f5 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:05.870164 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2f9c30-b555-4cfe-9438-099727094695 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:05.877987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-2d2f9c30-b555-4cfe-9438-099727094695 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:05.877987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:05.877987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:05.877987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:05.877987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:05.877987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:05.877987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:05.877987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:05.877987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:05.877987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:05.877987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:05.877987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:05.877987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:05.877987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:05.877987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:05.877987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:05.879653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:05.879653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:05.879653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:05.879653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:05.879653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:05.879653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:05.879653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:05.879653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:05.879653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:05.879653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:05.879653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:05.879653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:05.879653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:05.879653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:05.879653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:05.879653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:05.880426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:05.880426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:05.880426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:05.880426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:05.880426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:05.880426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:05.880426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:05.880426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:05.880426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:05.880426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:05.880426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:05.880426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:05.880426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 109/225] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:04:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:05.887380 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f9c30-b555-4cfe-9438-099727094695 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:05.937611 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c15484-c15c-4169-b870-72ce653b1a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:05.938267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c15484-c15c-4169-b870-72ce653b1a51 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:05.938527 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c15484-c15c-4169-b870-72ce653b1a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:05.938809 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c15484-c15c-4169-b870-72ce653b1a51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:05.966382 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c15484-c15c-4169-b870-72ce653b1a51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:05.971511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-82c15484-c15c-4169-b870-72ce653b1a51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:04:05.971511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:05.971511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:05.971511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:05.971511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:05.971511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:05.971511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:05.971511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:05.971511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:05.971511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:05.971511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:05.971511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:05.971511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:05.971511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:05.971511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:05.971511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:05.974151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:05.974151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:05.974151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:05.974151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:05.974151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:05.974151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:05.974151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:05.974151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:05.974151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:05.974151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:05.974151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:05.974151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:05.974151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:05.974151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:05.974151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:05.974151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:05.976532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:05.976532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:05.976532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:05.976532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:05.976532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:05.976532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:05.976532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:05.976532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:05.976532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:05.976532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:05.976532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:04:05.976532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:05.981758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 117/226] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:04:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:05.988251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c15484-c15c-4169-b870-72ce653b1a51 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:06.001257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:06.036914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2f9c30-b555-4cfe-9438-099727094695 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:06.076954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42a1af-b261-48b3-854b-7b2bf7ba2d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:06.077736 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42a1af-b261-48b3-854b-7b2bf7ba2d5f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:06.078017 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42a1af-b261-48b3-854b-7b2bf7ba2d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:06.079052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42a1af-b261-48b3-854b-7b2bf7ba2d5f None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:06.080429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42a1af-b261-48b3-854b-7b2bf7ba2d5f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:06.080762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42a1af-b261-48b3-854b-7b2bf7ba2d5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:06.082779 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42a1af-b261-48b3-854b-7b2bf7ba2d5f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:06.107148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 110/227] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:06.117501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42a1af-b261-48b3-854b-7b2bf7ba2d5f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:06.171596 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c15484-c15c-4169-b870-72ce653b1a51 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:06.215623 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e481534-e171-4a19-9099-163ab95fce87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:06.216742 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e481534-e171-4a19-9099-163ab95fce87 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:06.217808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e481534-e171-4a19-9099-163ab95fce87 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:06.218236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e481534-e171-4a19-9099-163ab95fce87 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:06.219176 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e481534-e171-4a19-9099-163ab95fce87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:06.219627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e481534-e171-4a19-9099-163ab95fce87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:06.221277 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e481534-e171-4a19-9099-163ab95fce87 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:06.232046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 118/228] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:06.241737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e481534-e171-4a19-9099-163ab95fce87 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:06.278103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:06.355682 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42a1af-b261-48b3-854b-7b2bf7ba2d5f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:06.356328 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:06.366024 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e481534-e171-4a19-9099-163ab95fce87 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:06.423363 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b8ccf-7d7c-4e8f-aa51-541c0b9e54e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:06.424230 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b8ccf-7d7c-4e8f-aa51-541c0b9e54e3 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:06.424684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b8ccf-7d7c-4e8f-aa51-541c0b9e54e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:06.425152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313b8ccf-7d7c-4e8f-aa51-541c0b9e54e3 None admin] PATH_INFO: `/v3/users/e1ee92ad9ee445fdb5e930f83bc05d9e` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:06.426989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b8ccf-7d7c-4e8f-aa51-541c0b9e54e3 None admin] RBAC: Authorizing `identity:get_user(user_id=e1ee92ad9ee445fdb5e930f83bc05d9e)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:06.428590 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313b8ccf-7d7c-4e8f-aa51-541c0b9e54e3 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:06.432000 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 111/229] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:06 2022] GET /identity/v3/users/e1ee92ad9ee445fdb5e930f83bc05d9e => generated 283 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:06.438245 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31f51d-aebf-4e8d-b3f5-b9ad012e8154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:06.439133 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31f51d-aebf-4e8d-b3f5-b9ad012e8154 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:06.439558 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31f51d-aebf-4e8d-b3f5-b9ad012e8154 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:06.440040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31f51d-aebf-4e8d-b3f5-b9ad012e8154 None admin] PATH_INFO: `/v3/users/e1ee92ad9ee445fdb5e930f83bc05d9e` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:06.442203 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b8ccf-7d7c-4e8f-aa51-541c0b9e54e3 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:06.443353 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31f51d-aebf-4e8d-b3f5-b9ad012e8154 None admin] RBAC: Authorizing `identity:get_user(user_id=e1ee92ad9ee445fdb5e930f83bc05d9e)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:06.447250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31f51d-aebf-4e8d-b3f5-b9ad012e8154 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:06.449275 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 119/230] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:06 2022] GET /identity/v3/users/e1ee92ad9ee445fdb5e930f83bc05d9e => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:06.457369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31f51d-aebf-4e8d-b3f5-b9ad012e8154 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:06.567957 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31f51d-aebf-4e8d-b3f5-b9ad012e8154 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:06.604557 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe1a10-400b-4128-99fd-e818c18ad720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:06.605476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe1a10-400b-4128-99fd-e818c18ad720 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:06.605906 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe1a10-400b-4128-99fd-e818c18ad720 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:06.606482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe1a10-400b-4128-99fd-e818c18ad720 None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:06.608105 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe1a10-400b-4128-99fd-e818c18ad720 None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:06.609762 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe1a10-400b-4128-99fd-e818c18ad720 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:06.611881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 120/231] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:06 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:06.621626 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe1a10-400b-4128-99fd-e818c18ad720 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:06.649422 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:06.668916 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313b8ccf-7d7c-4e8f-aa51-541c0b9e54e3 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:06.716370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:06.731820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82dcdfc-efce-4c5b-ad83-a334735837fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:06.732468 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82dcdfc-efce-4c5b-ad83-a334735837fa None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:06.732777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82dcdfc-efce-4c5b-ad83-a334735837fa None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:06.733049 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82dcdfc-efce-4c5b-ad83-a334735837fa None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:06.734349 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82dcdfc-efce-4c5b-ad83-a334735837fa None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:06.735799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82dcdfc-efce-4c5b-ad83-a334735837fa None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:06.737884 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 112/232] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:06 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:06.744198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82dcdfc-efce-4c5b-ad83-a334735837fa None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:06.827043 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe1a10-400b-4128-99fd-e818c18ad720 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:06.881944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960bdb15-adf6-4041-9423-56895eaeb108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:06.883071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bdb15-adf6-4041-9423-56895eaeb108 None admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:06.883557 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bdb15-adf6-4041-9423-56895eaeb108 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:06.884073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bdb15-adf6-4041-9423-56895eaeb108 None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92/users/e1ee92ad9ee445fdb5e930f83bc05d9e/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:06.888120 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bdb15-adf6-4041-9423-56895eaeb108 None admin] RBAC: Authorizing `identity:create_grant(project_id=c872c66e5ba64576ab2eac8ce92c8b92, user_id=e1ee92ad9ee445fdb5e930f83bc05d9e, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:06.889811 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960bdb15-adf6-4041-9423-56895eaeb108 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:06.911236 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82dcdfc-efce-4c5b-ad83-a334735837fa None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:06.914439 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 121/233] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:04:06 2022] PUT /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92/users/e1ee92ad9ee445fdb5e930f83bc05d9e/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:04:06.968244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805c4bc-e8c1-4787-8861-966b4e598461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:06.968889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805c4bc-e8c1-4787-8861-966b4e598461 None admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:06.969222 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805c4bc-e8c1-4787-8861-966b4e598461 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:06.969502 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805c4bc-e8c1-4787-8861-966b4e598461 None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92/users/e1ee92ad9ee445fdb5e930f83bc05d9e/roles/80e5528d7c8742f78e29e64c1216176c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:06.971702 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805c4bc-e8c1-4787-8861-966b4e598461 None admin] RBAC: Authorizing `identity:create_grant(project_id=c872c66e5ba64576ab2eac8ce92c8b92, user_id=e1ee92ad9ee445fdb5e930f83bc05d9e, role_id=80e5528d7c8742f78e29e64c1216176c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:06.973229 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805c4bc-e8c1-4787-8861-966b4e598461 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:06.992757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 113/234] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:04:06 2022] PUT /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92/users/e1ee92ad9ee445fdb5e930f83bc05d9e/roles/80e5528d7c8742f78e29e64c1216176c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:04:07.112322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6ec9c-9c52-4da0-a3c1-484ee436d54a None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:07.113415 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6ec9c-9c52-4da0-a3c1-484ee436d54a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:07.113526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6ec9c-9c52-4da0-a3c1-484ee436d54a None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:07.114869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 122/235] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:07.121910 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bba84-112f-452a-9d95-9a8a39593ddd None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:07.122414 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bba84-112f-452a-9d95-9a8a39593ddd None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:07.122485 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bba84-112f-452a-9d95-9a8a39593ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:07.144280 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-974bba84-112f-452a-9d95-9a8a39593ddd None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:07.145515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-974bba84-112f-452a-9d95-9a8a39593ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:07.186604 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda1564-61a7-4801-81c3-939651f77789 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:07.186942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda1564-61a7-4801-81c3-939651f77789 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:07.187224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fda1564-61a7-4801-81c3-939651f77789 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:07.188603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 123/236] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:04:07.195075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729622b3-8121-4a34-baee-f2a27406efe5 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:07.195359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729622b3-8121-4a34-baee-f2a27406efe5 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:07.195622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729622b3-8121-4a34-baee-f2a27406efe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:07.206073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:07.226003 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-729622b3-8121-4a34-baee-f2a27406efe5 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:07.227645 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-729622b3-8121-4a34-baee-f2a27406efe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:07.229508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 114/237] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:07.256140 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bba84-112f-452a-9d95-9a8a39593ddd None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:07.296886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:07.310645 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 124/238] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:04:07.336558 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729622b3-8121-4a34-baee-f2a27406efe5 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:07.393125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bba84-112f-452a-9d95-9a8a39593ddd None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:07.431150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b078d-57e7-44ec-a719-653bee98d63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:07.431806 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b078d-57e7-44ec-a719-653bee98d63a None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:07.432087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b078d-57e7-44ec-a719-653bee98d63a None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:07.432360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41b078d-57e7-44ec-a719-653bee98d63a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:07.434170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b078d-57e7-44ec-a719-653bee98d63a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:07.436184 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41b078d-57e7-44ec-a719-653bee98d63a None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:07.439031 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 115/239] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:07.445756 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b078d-57e7-44ec-a719-653bee98d63a None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:07.477832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729622b3-8121-4a34-baee-f2a27406efe5 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:07.517659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:07.532466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59cca5-a11d-4edd-9afd-b021024d7b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:07.533813 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59cca5-a11d-4edd-9afd-b021024d7b41 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:07.535026 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59cca5-a11d-4edd-9afd-b021024d7b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:07.535505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d59cca5-a11d-4edd-9afd-b021024d7b41 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:07.551048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d59cca5-a11d-4edd-9afd-b021024d7b41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:07.557935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-3d59cca5-a11d-4edd-9afd-b021024d7b41 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:04:07.557935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:07.557935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:07.557935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:07.557935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:07.557935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:07.557935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:07.557935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:07.557935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:07.557935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:07.557935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:07.557935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:07.557935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:07.557935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:07.557935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:07.557935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:07.558832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:07.558832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:07.558832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:07.558832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:07.558832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:07.558832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:07.558832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:07.558832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:07.558832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:07.558832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:07.558832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:07.558832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:07.558832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:07.558832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:07.558832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:07.558832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:07.559627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:07.559627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:07.559627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:07.559627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:07.559627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:07.559627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:07.559627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:07.559627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:07.559627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:07.559627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:07.559627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:04:07.559627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:07.562245 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 125/240] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:07.568534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59cca5-a11d-4edd-9afd-b021024d7b41 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:07.578936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41b078d-57e7-44ec-a719-653bee98d63a None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:07.630209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:07.641424 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6837e38-29f2-4366-9488-1d72a7c4102c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:07.642368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6837e38-29f2-4366-9488-1d72a7c4102c None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:07.642797 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6837e38-29f2-4366-9488-1d72a7c4102c None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:07.643228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6837e38-29f2-4366-9488-1d72a7c4102c None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:07.644167 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6837e38-29f2-4366-9488-1d72a7c4102c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:07.645793 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6837e38-29f2-4366-9488-1d72a7c4102c None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:07.692394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 116/241] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:04:07 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:04:07.694649 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d59cca5-a11d-4edd-9afd-b021024d7b41 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:07.740588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a053e2b4-98b2-43ed-a721-2adafe2dd361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:07.741244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053e2b4-98b2-43ed-a721-2adafe2dd361 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:07.741534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053e2b4-98b2-43ed-a721-2adafe2dd361 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:07.741809 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a053e2b4-98b2-43ed-a721-2adafe2dd361 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:07.742552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a053e2b4-98b2-43ed-a721-2adafe2dd361 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:07.742864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a053e2b4-98b2-43ed-a721-2adafe2dd361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:07.744405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a053e2b4-98b2-43ed-a721-2adafe2dd361 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:07.759193 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 126/242] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:07.765210 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6837e38-29f2-4366-9488-1d72a7c4102c None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:07.906363 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6837e38-29f2-4366-9488-1d72a7c4102c None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:07.916947 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:07.950563 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe43db4-3411-4943-b567-d65f5786294c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:07.951212 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe43db4-3411-4943-b567-d65f5786294c None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:07.951497 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe43db4-3411-4943-b567-d65f5786294c None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:07.951771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe43db4-3411-4943-b567-d65f5786294c None admin] PATH_INFO: `/v3/users/e4d728438a4f4a5ca342d3c8287183ea` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:07.955695 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63022a44-cbdf-4284-9508-9bebd37d1b52 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:07.955980 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63022a44-cbdf-4284-9508-9bebd37d1b52 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:07.956305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63022a44-cbdf-4284-9508-9bebd37d1b52 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:07.957882 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 127/243] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:04:07.962988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406e80c-a26f-406f-b7fb-11112da8f2d6 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:07.963256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406e80c-a26f-406f-b7fb-11112da8f2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:07.963534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406e80c-a26f-406f-b7fb-11112da8f2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:07.968314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe43db4-3411-4943-b567-d65f5786294c None admin] RBAC: Authorizing `identity:get_user(user_id=e4d728438a4f4a5ca342d3c8287183ea)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:07.970123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe43db4-3411-4943-b567-d65f5786294c None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:07.972409 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 117/244] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:07 2022] GET /identity/v3/users/e4d728438a4f4a5ca342d3c8287183ea => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:07.977769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe43db4-3411-4943-b567-d65f5786294c None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:07.985419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-1406e80c-a26f-406f-b7fb-11112da8f2d6 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:07.986617 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1406e80c-a26f-406f-b7fb-11112da8f2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:08.069661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 128/245] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:04:08.097859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406e80c-a26f-406f-b7fb-11112da8f2d6 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:08.119755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe43db4-3411-4943-b567-d65f5786294c None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:08.167748 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:08.181270 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa98775-9e4b-4c08-89cb-94a7e858a77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:08.182042 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa98775-9e4b-4c08-89cb-94a7e858a77f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:08.182692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa98775-9e4b-4c08-89cb-94a7e858a77f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:08.183069 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa98775-9e4b-4c08-89cb-94a7e858a77f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:08.233541 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:08.258265 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406e80c-a26f-406f-b7fb-11112da8f2d6 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:08.280504 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa98775-9e4b-4c08-89cb-94a7e858a77f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:08.282248 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa98775-9e4b-4c08-89cb-94a7e858a77f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:08.316120 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087cf0b-795e-4f0a-8c56-9b932d82a7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:08.317205 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087cf0b-795e-4f0a-8c56-9b932d82a7cc None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:08.317625 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087cf0b-795e-4f0a-8c56-9b932d82a7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:08.318342 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087cf0b-795e-4f0a-8c56-9b932d82a7cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:08.333607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087cf0b-795e-4f0a-8c56-9b932d82a7cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:08.338998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-f087cf0b-795e-4f0a-8c56-9b932d82a7cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:08.338998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:08.338998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:08.338998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:08.338998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:08.338998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:08.338998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:08.338998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:08.338998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:08.338998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:08.338998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:08.338998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:08.338998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:08.338998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:08.338998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:08.338998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:08.340042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:08.340042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:08.340042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:08.340042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:08.340042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:08.340042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:08.340042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:08.340042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:08.340042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:08.340042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:08.340042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:08.340042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:08.340042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:08.340042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:08.340042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:08.340042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:08.341737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:08.341737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:08.341737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:08.341737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:08.341737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:08.341737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:08.341737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:08.341737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:08.341737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:08.341737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:08.341737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:08.341737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:08.345038 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 129/246] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:08.349435 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087cf0b-795e-4f0a-8c56-9b932d82a7cc None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:08.404251 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 118/247] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 02:04:08.409698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa98775-9e4b-4c08-89cb-94a7e858a77f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:08.465097 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087cf0b-795e-4f0a-8c56-9b932d82a7cc None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:08.467639 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:08.485328 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:08.502694 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7381a88-2184-40f0-904a-a4616c10a999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:08.503877 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7381a88-2184-40f0-904a-a4616c10a999 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:08.504155 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7381a88-2184-40f0-904a-a4616c10a999 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:08.504424 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7381a88-2184-40f0-904a-a4616c10a999 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:08.505184 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7381a88-2184-40f0-904a-a4616c10a999 None admin] RBAC: Adding query filter params (name=member) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:08.505487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7381a88-2184-40f0-904a-a4616c10a999 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:08.507199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7381a88-2184-40f0-904a-a4616c10a999 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:08.516172 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 130/248] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:08.520171 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa98775-9e4b-4c08-89cb-94a7e858a77f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:08.521674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7381a88-2184-40f0-904a-a4616c10a999 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:08.589443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95289ade-ff12-4738-843e-2d3d0f2b163f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:08.590426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95289ade-ff12-4738-843e-2d3d0f2b163f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:08.590898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95289ade-ff12-4738-843e-2d3d0f2b163f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:08.591374 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95289ade-ff12-4738-843e-2d3d0f2b163f None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:08.593368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95289ade-ff12-4738-843e-2d3d0f2b163f None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:08.595107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95289ade-ff12-4738-843e-2d3d0f2b163f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:08.602119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 119/249] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:08 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:08.603286 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95289ade-ff12-4738-843e-2d3d0f2b163f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:08.629118 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7381a88-2184-40f0-904a-a4616c10a999 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:08.680388 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba746aa-1864-4367-b9e2-c68a2beda2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:08.681127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba746aa-1864-4367-b9e2-c68a2beda2ba None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:08.681406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba746aa-1864-4367-b9e2-c68a2beda2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:08.681688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba746aa-1864-4367-b9e2-c68a2beda2ba None admin] PATH_INFO: `/v3/users/6ed69a73796d4f67b1a7458a294afa70` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:08.683999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba746aa-1864-4367-b9e2-c68a2beda2ba None admin] RBAC: Authorizing `identity:get_user(user_id=6ed69a73796d4f67b1a7458a294afa70)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:08.685749 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba746aa-1864-4367-b9e2-c68a2beda2ba None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:08.692008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 131/250] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:08 2022] GET /identity/v3/users/6ed69a73796d4f67b1a7458a294afa70 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:08.694332 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba746aa-1864-4367-b9e2-c68a2beda2ba None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:08.753359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:08.838504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba746aa-1864-4367-b9e2-c68a2beda2ba None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:08.847619 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95289ade-ff12-4738-843e-2d3d0f2b163f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:08.879605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e6c5c-91d9-4715-ae92-3bcf1e49a989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:08.880267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e6c5c-91d9-4715-ae92-3bcf1e49a989 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:08.880546 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e6c5c-91d9-4715-ae92-3bcf1e49a989 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:08.880843 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e6c5c-91d9-4715-ae92-3bcf1e49a989 None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:08.885289 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e6c5c-91d9-4715-ae92-3bcf1e49a989 None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:08.886754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e6c5c-91d9-4715-ae92-3bcf1e49a989 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:08.888517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 132/251] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:08 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:08.898588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e6c5c-91d9-4715-ae92-3bcf1e49a989 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:08.922662 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f73952-8f7f-4d3a-91ea-b06f3dbb2555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:08.923689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f73952-8f7f-4d3a-91ea-b06f3dbb2555 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:08.923972 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f73952-8f7f-4d3a-91ea-b06f3dbb2555 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:08.924246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f73952-8f7f-4d3a-91ea-b06f3dbb2555 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:08.925803 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f73952-8f7f-4d3a-91ea-b06f3dbb2555 None admin] RBAC: Adding query filter params (role.id=9e8771d0146c45e6afaf7915e215ac91, scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92, user.id=e4d728438a4f4a5ca342d3c8287183ea) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:08.926116 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f73952-8f7f-4d3a-91ea-b06f3dbb2555 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:08.927572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f73952-8f7f-4d3a-91ea-b06f3dbb2555 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:08.932568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:08.937836 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 120/252] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:08 2022] GET /identity/v3/role_assignments?user.id=e4d728438a4f4a5ca342d3c8287183ea&scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92&role.id=9e8771d0146c45e6afaf7915e215ac91 => generated 259 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:09.026159 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e6c5c-91d9-4715-ae92-3bcf1e49a989 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:09.027842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:09.064180 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bff713-be8a-4b28-95ca-8cff49f8f900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:09.064976 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bff713-be8a-4b28-95ca-8cff49f8f900 None admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:09.065256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bff713-be8a-4b28-95ca-8cff49f8f900 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:09.065529 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bff713-be8a-4b28-95ca-8cff49f8f900 None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92/users/6ed69a73796d4f67b1a7458a294afa70/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:09.067766 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bff713-be8a-4b28-95ca-8cff49f8f900 None admin] RBAC: Authorizing `identity:create_grant(project_id=c872c66e5ba64576ab2eac8ce92c8b92, user_id=6ed69a73796d4f67b1a7458a294afa70, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:09.069273 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bff713-be8a-4b28-95ca-8cff49f8f900 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:09.090742 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 133/253] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:04:08 2022] PUT /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92/users/6ed69a73796d4f67b1a7458a294afa70/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:04:09.909030 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459bc73-6eab-4c16-9d33-bb6d360535aa None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:09.910064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459bc73-6eab-4c16-9d33-bb6d360535aa None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:09.910064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459bc73-6eab-4c16-9d33-bb6d360535aa None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:09.911455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 121/254] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:09.917133 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd71897-aa49-4c80-ade4-ac8242c7a9f8 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:09.917594 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd71897-aa49-4c80-ade4-ac8242c7a9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:09.917690 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd71897-aa49-4c80-ade4-ac8242c7a9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:09.943528 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-bcd71897-aa49-4c80-ade4-ac8242c7a9f8 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:09.944770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-bcd71897-aa49-4c80-ade4-ac8242c7a9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:10.019580 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 134/255] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:10.066150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459bc73-6eab-4c16-9d33-bb6d360535aa None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:10.174046 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459bc73-6eab-4c16-9d33-bb6d360535aa None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:10.208860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e63ab-3a6d-44d1-bcb2-1d5bbd7a6637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:10.209501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e63ab-3a6d-44d1-bcb2-1d5bbd7a6637 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:10.209779 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e63ab-3a6d-44d1-bcb2-1d5bbd7a6637 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:10.210056 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e63ab-3a6d-44d1-bcb2-1d5bbd7a6637 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:10.221067 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e63ab-3a6d-44d1-bcb2-1d5bbd7a6637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:10.227028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-1d5e63ab-3a6d-44d1-bcb2-1d5bbd7a6637 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:04:10.227028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:10.227028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:10.227028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:10.227028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:10.227028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:10.227028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:10.227028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:10.227028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:10.227028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:10.227028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:10.227028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:10.227028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:10.227028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:10.227028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:10.227028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:10.228390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:10.228390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:10.228390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:10.228390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:10.228390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:10.228390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:10.228390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:10.228390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:10.228390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:10.228390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:10.228390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:10.228390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:10.228390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:10.228390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:10.228390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:10.228390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:10.230640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:10.230640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:10.230640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:10.230640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:10.230640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:10.230640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:10.230640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:10.230640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:10.230640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:10.230640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:10.230640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:04:10.230640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:10.230640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 122/256] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:04:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:10.241656 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd71897-aa49-4c80-ade4-ac8242c7a9f8 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:10.369305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:10.376910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd71897-aa49-4c80-ade4-ac8242c7a9f8 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:10.417260 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbb8ed-2687-4a6b-982d-6e20cad8480f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:10.417882 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbb8ed-2687-4a6b-982d-6e20cad8480f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:10.418159 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbb8ed-2687-4a6b-982d-6e20cad8480f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:10.418431 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbb8ed-2687-4a6b-982d-6e20cad8480f None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:10.419159 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cbb8ed-2687-4a6b-982d-6e20cad8480f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:10.419522 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cbb8ed-2687-4a6b-982d-6e20cad8480f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:10.421035 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cbb8ed-2687-4a6b-982d-6e20cad8480f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:10.431233 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 135/257] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:10.434234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162f7af-302d-4459-adda-055bd4b70e9e None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:10.434690 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162f7af-302d-4459-adda-055bd4b70e9e None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:10.435130 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4162f7af-302d-4459-adda-055bd4b70e9e None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:10.437069 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 123/258] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:10.438022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbb8ed-2687-4a6b-982d-6e20cad8480f None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:10.442282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2708e8-1558-44a4-8d26-b30ea6093beb None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:10.442563 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2708e8-1558-44a4-8d26-b30ea6093beb None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:10.442833 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2708e8-1558-44a4-8d26-b30ea6093beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:10.473706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-8a2708e8-1558-44a4-8d26-b30ea6093beb None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:10.475230 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-8a2708e8-1558-44a4-8d26-b30ea6093beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:10.566812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbb8ed-2687-4a6b-982d-6e20cad8480f None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:10.606593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84557ba6-3762-4da5-bd1e-57e6b12ea89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:10.607967 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84557ba6-3762-4da5-bd1e-57e6b12ea89f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:10.608784 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84557ba6-3762-4da5-bd1e-57e6b12ea89f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:10.610602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84557ba6-3762-4da5-bd1e-57e6b12ea89f None admin] PATH_INFO: `/v3/users/e1ee92ad9ee445fdb5e930f83bc05d9e` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:10.613801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84557ba6-3762-4da5-bd1e-57e6b12ea89f None admin] RBAC: Authorizing `identity:get_user(user_id=e1ee92ad9ee445fdb5e930f83bc05d9e)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:10.615488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84557ba6-3762-4da5-bd1e-57e6b12ea89f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:10.623109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 136/259] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:10 2022] GET /identity/v3/users/e1ee92ad9ee445fdb5e930f83bc05d9e => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:10.629828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84557ba6-3762-4da5-bd1e-57e6b12ea89f None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:10.656998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:10.689073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:10.690796 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 124/260] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:04:10.730549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2708e8-1558-44a4-8d26-b30ea6093beb None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:10.824747 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84557ba6-3762-4da5-bd1e-57e6b12ea89f None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:10.838748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2708e8-1558-44a4-8d26-b30ea6093beb None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:10.874148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd9211-fdb1-4b6d-ae92-9e09cb98af1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:10.874795 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd9211-fdb1-4b6d-ae92-9e09cb98af1f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:10.875073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd9211-fdb1-4b6d-ae92-9e09cb98af1f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:10.875348 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd9211-fdb1-4b6d-ae92-9e09cb98af1f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:10.886756 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd9211-fdb1-4b6d-ae92-9e09cb98af1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:10.890258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdd6bc8-1b70-48d8-9a0a-fe4a9100fbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:10.890903 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd6bc8-1b70-48d8-9a0a-fe4a9100fbfb None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:10.891187 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd6bc8-1b70-48d8-9a0a-fe4a9100fbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:10.891463 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdd6bc8-1b70-48d8-9a0a-fe4a9100fbfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:10.892125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-e5fd9211-fdb1-4b6d-ae92-9e09cb98af1f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:10.892125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:10.892125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:10.892125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:10.892125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:10.892125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:10.892125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:10.892125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:10.892125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:10.892125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:10.892125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:10.892125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:10.892125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:10.892125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:10.892125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:10.892125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:10.893097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:10.893097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:10.893097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:10.893097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:10.893097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:10.893097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:10.893097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:10.893097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:10.893097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:10.893097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:10.893097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:10.893097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:10.893097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:10.893097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:10.893097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:10.893097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:10.894420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:10.894420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:10.894420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:10.894420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:10.894420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:10.894420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:10.894420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:10.894420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:10.894420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:10.894420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:10.894420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:10.894420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:10.894420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 125/261] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:04:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:10.902043 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd9211-fdb1-4b6d-ae92-9e09cb98af1f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:11.017070 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:11.061491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd6bc8-1b70-48d8-9a0a-fe4a9100fbfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:11.062985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdd6bc8-1b70-48d8-9a0a-fe4a9100fbfb None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:11.114870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:11.119802 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd9211-fdb1-4b6d-ae92-9e09cb98af1f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:11.194599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96036a-43f9-45d9-809e-b7e2558b8b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:11.195237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96036a-43f9-45d9-809e-b7e2558b8b94 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:11.195512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96036a-43f9-45d9-809e-b7e2558b8b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:11.195785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96036a-43f9-45d9-809e-b7e2558b8b94 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:11.196523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96036a-43f9-45d9-809e-b7e2558b8b94 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:11.196846 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96036a-43f9-45d9-809e-b7e2558b8b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:11.198248 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96036a-43f9-45d9-809e-b7e2558b8b94 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:11.216478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 126/262] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:11.218099 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7296f5-75ba-402b-998e-ffc8c1aeb6ae None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:11.218586 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7296f5-75ba-402b-998e-ffc8c1aeb6ae None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:11.219080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7296f5-75ba-402b-998e-ffc8c1aeb6ae None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:11.220915 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 127/263] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:11.224062 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7296f5-75ba-402b-998e-ffc8c1aeb6ae None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:11.299330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 137/264] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 670 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 02:04:11.301137 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b0527-b49a-48b2-9ec7-8f558967a25a None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:11.301424 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b0527-b49a-48b2-9ec7-8f558967a25a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:11.301698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b0527-b49a-48b2-9ec7-8f558967a25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:11.340977 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7296f5-75ba-402b-998e-ffc8c1aeb6ae None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:11.343377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-d84b0527-b49a-48b2-9ec7-8f558967a25a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:11.344881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d84b0527-b49a-48b2-9ec7-8f558967a25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:11.354280 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:11.386567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677012fa-7f07-4ccb-a1fc-967128d84ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:11.387487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677012fa-7f07-4ccb-a1fc-967128d84ada None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:11.387915 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677012fa-7f07-4ccb-a1fc-967128d84ada None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:11.388334 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677012fa-7f07-4ccb-a1fc-967128d84ada None admin] PATH_INFO: `/v3/users/e1ee92ad9ee445fdb5e930f83bc05d9e` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:11.389730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677012fa-7f07-4ccb-a1fc-967128d84ada None admin] RBAC: Authorizing `identity:get_user(user_id=e1ee92ad9ee445fdb5e930f83bc05d9e)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:11.391894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677012fa-7f07-4ccb-a1fc-967128d84ada None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:11.394506 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 128/265] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:11 2022] GET /identity/v3/users/e1ee92ad9ee445fdb5e930f83bc05d9e => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:11.395752 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677012fa-7f07-4ccb-a1fc-967128d84ada None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:11.420779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 138/266] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:11.422542 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6f587-9638-486d-a8ce-c348d13b5ecc None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:11.423029 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6f587-9638-486d-a8ce-c348d13b5ecc None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:11.423517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6f587-9638-486d-a8ce-c348d13b5ecc None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:11.425696 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 139/267] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:11.426623 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6f587-9638-486d-a8ce-c348d13b5ecc None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:11.488089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:11.524565 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677012fa-7f07-4ccb-a1fc-967128d84ada None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:11.541958 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a6f587-9638-486d-a8ce-c348d13b5ecc None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:11.572416 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b605713-c3bb-44c5-a177-4bfa0fb465cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:11.573012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b605713-c3bb-44c5-a177-4bfa0fb465cf None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:11.573292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b605713-c3bb-44c5-a177-4bfa0fb465cf None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:11.573566 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b605713-c3bb-44c5-a177-4bfa0fb465cf None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:11.575229 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b605713-c3bb-44c5-a177-4bfa0fb465cf None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:11.576730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b605713-c3bb-44c5-a177-4bfa0fb465cf None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:11.578705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 129/268] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:11 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:11.579688 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7729069-b94e-490a-9d0e-5ef62c64d9fd None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:11.579962 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7729069-b94e-490a-9d0e-5ef62c64d9fd None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:11.580239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7729069-b94e-490a-9d0e-5ef62c64d9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:11.588059 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab3578-3ac5-424f-b462-7489f3dd4854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:11.588705 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab3578-3ac5-424f-b462-7489f3dd4854 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:11.588989 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab3578-3ac5-424f-b462-7489f3dd4854 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:11.589263 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab3578-3ac5-424f-b462-7489f3dd4854 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:11.599570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-e7729069-b94e-490a-9d0e-5ef62c64d9fd None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:11.600780 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e7729069-b94e-490a-9d0e-5ef62c64d9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:11.644521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:11.669737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab3578-3ac5-424f-b462-7489f3dd4854 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:11.671462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab3578-3ac5-424f-b462-7489f3dd4854 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:11.672853 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 130/269] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:11.674052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7729069-b94e-490a-9d0e-5ef62c64d9fd None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:11.739583 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:11.778273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7729069-b94e-490a-9d0e-5ef62c64d9fd None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:11.789719 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 140/270] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:11.795484 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511ba3d-ecc9-4e76-aa66-797def88c186 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:11.795772 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511ba3d-ecc9-4e76-aa66-797def88c186 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:11.796051 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0511ba3d-ecc9-4e76-aa66-797def88c186 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:11.800880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 141/271] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:11.801542 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511ba3d-ecc9-4e76-aa66-797def88c186 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:11.816760 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75b625-6605-47e3-a269-c84195ae0f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:11.817418 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75b625-6605-47e3-a269-c84195ae0f80 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:11.817694 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75b625-6605-47e3-a269-c84195ae0f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:11.817971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75b625-6605-47e3-a269-c84195ae0f80 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:11.819114 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75b625-6605-47e3-a269-c84195ae0f80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:11.820581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75b625-6605-47e3-a269-c84195ae0f80 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:11.822363 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 131/272] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:11.823021 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75b625-6605-47e3-a269-c84195ae0f80 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:11.871498 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:11.919354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0511ba3d-ecc9-4e76-aa66-797def88c186 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:11.935727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75b625-6605-47e3-a269-c84195ae0f80 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:11.959810 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab813c9-bc8d-431a-bc45-dc0a01146bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:11.960903 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab813c9-bc8d-431a-bc45-dc0a01146bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:11.961180 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab813c9-bc8d-431a-bc45-dc0a01146bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:11.961927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab813c9-bc8d-431a-bc45-dc0a01146bd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:11.962728 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab813c9-bc8d-431a-bc45-dc0a01146bd9 None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92, user.id=e1ee92ad9ee445fdb5e930f83bc05d9e) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:11.963187 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab813c9-bc8d-431a-bc45-dc0a01146bd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:11.965293 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab813c9-bc8d-431a-bc45-dc0a01146bd9 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:11.980647 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 142/273] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:11 2022] GET /identity/v3/role_assignments?user.id=e1ee92ad9ee445fdb5e930f83bc05d9e&scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92&role.id=35205394c8af4c52a61b12f92ba16644 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:11.980647 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab813c9-bc8d-431a-bc45-dc0a01146bd9 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:11.982969 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb853b-c4e0-4860-9b17-21a65b4ba494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:11.983778 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb853b-c4e0-4860-9b17-21a65b4ba494 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:11.984131 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb853b-c4e0-4860-9b17-21a65b4ba494 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:11.984480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb853b-c4e0-4860-9b17-21a65b4ba494 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:11.998320 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fb853b-c4e0-4860-9b17-21a65b4ba494 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:12.002724 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:12.005669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-10fb853b-c4e0-4860-9b17-21a65b4ba494 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:12.005669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:12.005669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:12.005669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:12.005669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:12.005669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:12.005669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:12.005669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:12.005669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:12.005669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:12.005669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:12.005669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:12.005669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:12.005669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:12.005669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:12.005669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:12.006698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:12.006698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:12.006698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:12.006698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:12.006698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:12.006698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:12.006698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:12.006698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:12.006698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:12.006698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:12.006698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:12.006698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:12.006698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:12.006698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:12.006698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:12.006698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:12.007457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:12.007457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:12.007457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:12.007457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:12.007457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:12.007457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:12.007457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:12.007457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:12.007457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:12.007457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:12.007457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:12.007457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:12.007457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 132/274] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:12.014895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afc578-75ac-4294-bbee-89a6be3f5b34 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:12.015179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afc578-75ac-4294-bbee-89a6be3f5b34 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:12.015604 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19afc578-75ac-4294-bbee-89a6be3f5b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:12.037602 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-19afc578-75ac-4294-bbee-89a6be3f5b34 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:12.039032 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-19afc578-75ac-4294-bbee-89a6be3f5b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:12.119433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab813c9-bc8d-431a-bc45-dc0a01146bd9 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:12.127950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 133/275] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:12.129143 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afc578-75ac-4294-bbee-89a6be3f5b34 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:12.160755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c459b02-3dcf-461a-b138-bb630e70815e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:12.161543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c459b02-3dcf-461a-b138-bb630e70815e None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:12.161946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c459b02-3dcf-461a-b138-bb630e70815e None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:12.162289 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c459b02-3dcf-461a-b138-bb630e70815e None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:12.164468 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c459b02-3dcf-461a-b138-bb630e70815e None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:12.166175 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c459b02-3dcf-461a-b138-bb630e70815e None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:12.168500 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:12.169442 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 143/276] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:11 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:12.171295 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c459b02-3dcf-461a-b138-bb630e70815e None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:12.252137 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19afc578-75ac-4294-bbee-89a6be3f5b34 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:12.283824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:12.301758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c459b02-3dcf-461a-b138-bb630e70815e None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:12.302041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d212fe-d93b-4fb3-903d-b65008554b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:12.302430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d212fe-d93b-4fb3-903d-b65008554b1f None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:12.302710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d212fe-d93b-4fb3-903d-b65008554b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:12.302983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d212fe-d93b-4fb3-903d-b65008554b1f None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:12.303668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d212fe-d93b-4fb3-903d-b65008554b1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:12.305275 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d212fe-d93b-4fb3-903d-b65008554b1f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:12.327080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 134/277] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:04:12 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:04:12.327978 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d212fe-d93b-4fb3-903d-b65008554b1f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:12.341045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adb99b-8bd4-42e5-b29d-0ef3b9234a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:12.342117 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adb99b-8bd4-42e5-b29d-0ef3b9234a6c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:12.342406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adb99b-8bd4-42e5-b29d-0ef3b9234a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:12.342679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2adb99b-8bd4-42e5-b29d-0ef3b9234a6c None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:12.343444 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adb99b-8bd4-42e5-b29d-0ef3b9234a6c None admin] RBAC: Adding query filter params (name=member) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:12.343748 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adb99b-8bd4-42e5-b29d-0ef3b9234a6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:12.345229 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2adb99b-8bd4-42e5-b29d-0ef3b9234a6c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:12.361862 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 144/278] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:12.361862 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adb99b-8bd4-42e5-b29d-0ef3b9234a6c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:12.442642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:12.456761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d212fe-d93b-4fb3-903d-b65008554b1f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:12.482360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2adb99b-8bd4-42e5-b29d-0ef3b9234a6c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:12.494669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af8a03-b99e-4bd4-823b-1bb3303cbe9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:12.495317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af8a03-b99e-4bd4-823b-1bb3303cbe9c None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:12.495610 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af8a03-b99e-4bd4-823b-1bb3303cbe9c None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:12.495889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af8a03-b99e-4bd4-823b-1bb3303cbe9c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:12.507737 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af8a03-b99e-4bd4-823b-1bb3303cbe9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:12.513377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-25af8a03-b99e-4bd4-823b-1bb3303cbe9c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:04:12.513377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:12.513377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:12.513377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:12.513377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:12.513377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:12.513377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:12.513377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:12.513377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:12.513377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:12.513377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:12.513377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:12.513377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:12.513377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:12.513377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:12.513377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:12.514269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:12.514269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:12.514269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:12.514269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:12.514269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:12.514269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:12.514269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:12.514269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:12.514269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:12.514269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:12.514269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:12.514269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:12.514269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:12.514269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:12.514269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:12.514269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:12.515117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:12.515117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:12.515117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:12.515117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:12.515117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:12.515117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:12.515117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:12.515117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:12.515117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:12.515117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:12.515117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:04:12.515117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:12.515117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 135/279] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:12.515818 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14ab62-91b2-4d92-97e5-649e07492498 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:12.516023 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14ab62-91b2-4d92-97e5-649e07492498 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:12.516304 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14ab62-91b2-4d92-97e5-649e07492498 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:12.521905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 136/280] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:12.522731 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14ab62-91b2-4d92-97e5-649e07492498 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:12.530407 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39990d1-fc4b-47d1-a030-b33cdb388e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:12.531050 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39990d1-fc4b-47d1-a030-b33cdb388e3a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:12.531334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39990d1-fc4b-47d1-a030-b33cdb388e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:12.531741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39990d1-fc4b-47d1-a030-b33cdb388e3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:12.532674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39990d1-fc4b-47d1-a030-b33cdb388e3a None admin] RBAC: Adding query filter params (role.id=80e5528d7c8742f78e29e64c1216176c, scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92, user.id=e1ee92ad9ee445fdb5e930f83bc05d9e) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:12.532980 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39990d1-fc4b-47d1-a030-b33cdb388e3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:12.534467 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39990d1-fc4b-47d1-a030-b33cdb388e3a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:12.542777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 145/281] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:12 2022] GET /identity/v3/role_assignments?user.id=e1ee92ad9ee445fdb5e930f83bc05d9e&scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92&role.id=80e5528d7c8742f78e29e64c1216176c => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:12.543072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39990d1-fc4b-47d1-a030-b33cdb388e3a None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:12.558465 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:12.681443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14ab62-91b2-4d92-97e5-649e07492498 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:12.687688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39990d1-fc4b-47d1-a030-b33cdb388e3a None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:12.725274 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f465a-bc0b-4b06-b30e-ce6258f86f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:12.725842 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f465a-bc0b-4b06-b30e-ce6258f86f3b None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:12.726119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f465a-bc0b-4b06-b30e-ce6258f86f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:12.726388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f465a-bc0b-4b06-b30e-ce6258f86f3b None admin] PATH_INFO: `/v3/users/e38c71c6201a427c8cb2a0d0dbcfa058` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:12.731669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748bb15b-85e1-469d-b7b0-f61b6d696112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:12.732603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bb15b-85e1-469d-b7b0-f61b6d696112 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:12.733093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bb15b-85e1-469d-b7b0-f61b6d696112 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:12.733545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bb15b-85e1-469d-b7b0-f61b6d696112 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:12.734451 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748bb15b-85e1-469d-b7b0-f61b6d696112 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:12.734998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748bb15b-85e1-469d-b7b0-f61b6d696112 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:12.736585 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748bb15b-85e1-469d-b7b0-f61b6d696112 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:12.743874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f465a-bc0b-4b06-b30e-ce6258f86f3b None admin] RBAC: Authorizing `identity:get_user(user_id=e38c71c6201a427c8cb2a0d0dbcfa058)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:12.745378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f465a-bc0b-4b06-b30e-ce6258f86f3b None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:12.747126 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 146/282] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:12.747334 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 137/283] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:12 2022] GET /identity/v3/users/e38c71c6201a427c8cb2a0d0dbcfa058 => generated 293 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:12.748776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967d5bc-b826-48ee-9d72-eb1eceef70b5 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:12.749251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967d5bc-b826-48ee-9d72-eb1eceef70b5 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:12.749703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967d5bc-b826-48ee-9d72-eb1eceef70b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:12.753248 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f465a-bc0b-4b06-b30e-ce6258f86f3b None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:12.765644 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:12.772031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-3967d5bc-b826-48ee-9d72-eb1eceef70b5 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:12.773577 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-3967d5bc-b826-48ee-9d72-eb1eceef70b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:12.849676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 147/284] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:12.850807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967d5bc-b826-48ee-9d72-eb1eceef70b5 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:12.853380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:12.870589 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f465a-bc0b-4b06-b30e-ce6258f86f3b None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:12.911242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787ff28-b011-4688-9d2c-bc72b0de44e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:12.911971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787ff28-b011-4688-9d2c-bc72b0de44e8 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:12.912327 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787ff28-b011-4688-9d2c-bc72b0de44e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:12.912721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787ff28-b011-4688-9d2c-bc72b0de44e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:12.913756 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787ff28-b011-4688-9d2c-bc72b0de44e8 None admin] RBAC: Adding query filter params (role.id=8fc75c351a0349e294deeed1ff391c4c, user.id=e38c71c6201a427c8cb2a0d0dbcfa058, scope.system=all) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:12.914145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787ff28-b011-4688-9d2c-bc72b0de44e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:12.915864 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8787ff28-b011-4688-9d2c-bc72b0de44e8 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:12.922323 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 138/285] 10.222.0.45 () {56 vars in 1308 bytes} [Thu Aug 4 02:04:12 2022] GET /identity/v3/role_assignments?user.id=e38c71c6201a427c8cb2a0d0dbcfa058&scope.system=all&role.id=8fc75c351a0349e294deeed1ff391c4c => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:12.923354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787ff28-b011-4688-9d2c-bc72b0de44e8 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:12.964926 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967d5bc-b826-48ee-9d72-eb1eceef70b5 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:13.002598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f0d3f-6f7f-46f5-a976-40eb7fe66584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:13.003624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f0d3f-6f7f-46f5-a976-40eb7fe66584 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:13.004119 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f0d3f-6f7f-46f5-a976-40eb7fe66584 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:13.004597 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f0d3f-6f7f-46f5-a976-40eb7fe66584 None admin] PATH_INFO: `/v3/users/e4d728438a4f4a5ca342d3c8287183ea` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:13.005815 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:13.006382 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f0d3f-6f7f-46f5-a976-40eb7fe66584 None admin] RBAC: Authorizing `identity:get_user(user_id=e4d728438a4f4a5ca342d3c8287183ea)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:13.008099 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5f0d3f-6f7f-46f5-a976-40eb7fe66584 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:13.009991 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 148/286] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:12 2022] GET /identity/v3/users/e4d728438a4f4a5ca342d3c8287183ea => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:13.020672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f0d3f-6f7f-46f5-a976-40eb7fe66584 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:13.045500 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8787ff28-b011-4688-9d2c-bc72b0de44e8 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:13.085021 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f978f-2ee0-4fe6-854c-6f37d8e68785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:13.085682 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f978f-2ee0-4fe6-854c-6f37d8e68785 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:13.085964 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f978f-2ee0-4fe6-854c-6f37d8e68785 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:13.086559 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1f978f-2ee0-4fe6-854c-6f37d8e68785 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:13.100204 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1f978f-2ee0-4fe6-854c-6f37d8e68785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:13.105671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-dc1f978f-2ee0-4fe6-854c-6f37d8e68785 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:13.105671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:13.105671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:13.105671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:13.105671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:13.105671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:13.105671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:13.105671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:13.105671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:13.105671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:13.105671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:13.105671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:13.105671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:13.105671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:13.105671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:13.105671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:13.106599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:13.106599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:13.106599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:13.106599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:13.106599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:13.106599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:13.106599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:13.106599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:13.106599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:13.106599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:13.106599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:13.106599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:13.106599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:13.106599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:13.106599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:13.106599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:13.107407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:13.107407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:13.107407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:13.107407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:13.107407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:13.107407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:13.107407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:13.107407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:13.107407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:13.107407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:13.107407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:13.107407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:13.107407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 139/287] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:13.115128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f978f-2ee0-4fe6-854c-6f37d8e68785 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:13.229244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1f978f-2ee0-4fe6-854c-6f37d8e68785 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:13.264867 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:13.272510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106a7c2-1ed1-4ae3-80d9-19854118eb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:13.273909 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106a7c2-1ed1-4ae3-80d9-19854118eb42 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:13.274193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106a7c2-1ed1-4ae3-80d9-19854118eb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:13.274472 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106a7c2-1ed1-4ae3-80d9-19854118eb42 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:13.275217 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106a7c2-1ed1-4ae3-80d9-19854118eb42 None admin] RBAC: Adding query filter params (name=member) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:13.275527 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106a7c2-1ed1-4ae3-80d9-19854118eb42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:13.277167 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106a7c2-1ed1-4ae3-80d9-19854118eb42 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:13.282596 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:13.289716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 140/288] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:13.296162 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106a7c2-1ed1-4ae3-80d9-19854118eb42 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:13.366593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5f0d3f-6f7f-46f5-a976-40eb7fe66584 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:13.410890 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106a7c2-1ed1-4ae3-80d9-19854118eb42 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:13.449853 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc261c4-83db-4b96-b816-65bd37e1c938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:13.450993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc261c4-83db-4b96-b816-65bd37e1c938 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:13.451454 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc261c4-83db-4b96-b816-65bd37e1c938 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:13.452166 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc261c4-83db-4b96-b816-65bd37e1c938 None admin] PATH_INFO: `/v3/users/6ed69a73796d4f67b1a7458a294afa70` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:13.453970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc261c4-83db-4b96-b816-65bd37e1c938 None admin] RBAC: Authorizing `identity:get_user(user_id=6ed69a73796d4f67b1a7458a294afa70)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:13.455630 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc261c4-83db-4b96-b816-65bd37e1c938 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:13.458185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 141/289] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:13 2022] GET /identity/v3/users/6ed69a73796d4f67b1a7458a294afa70 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:13.463851 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc261c4-83db-4b96-b816-65bd37e1c938 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:13.468968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c69da3-5f9a-4400-a09c-99b7ff499fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:13.469680 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c69da3-5f9a-4400-a09c-99b7ff499fbc None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:13.469960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c69da3-5f9a-4400-a09c-99b7ff499fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:13.470234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c69da3-5f9a-4400-a09c-99b7ff499fbc None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:13.477386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c69da3-5f9a-4400-a09c-99b7ff499fbc None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:13.478845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c69da3-5f9a-4400-a09c-99b7ff499fbc None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:13.484650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 149/290] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:13 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 463 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:13.491978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c69da3-5f9a-4400-a09c-99b7ff499fbc None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:13.541107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:13.589204 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc261c4-83db-4b96-b816-65bd37e1c938 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:13.627094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915301d-e5ef-4d8e-bc16-0e959a69f842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:13.628583 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915301d-e5ef-4d8e-bc16-0e959a69f842 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:13.629094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915301d-e5ef-4d8e-bc16-0e959a69f842 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:13.629586 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915301d-e5ef-4d8e-bc16-0e959a69f842 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:13.703437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915301d-e5ef-4d8e-bc16-0e959a69f842 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:13.705100 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915301d-e5ef-4d8e-bc16-0e959a69f842 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:13.777013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:13.819792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 142/291] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:13.823706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c69da3-5f9a-4400-a09c-99b7ff499fbc None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:13.825228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915301d-e5ef-4d8e-bc16-0e959a69f842 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:13.902068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92ed72-7f3e-452b-abc4-ba5bd0da8116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:13.902920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92ed72-7f3e-452b-abc4-ba5bd0da8116 None admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:13.903321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92ed72-7f3e-452b-abc4-ba5bd0da8116 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:13.903700 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92ed72-7f3e-452b-abc4-ba5bd0da8116 None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92/users/e4d728438a4f4a5ca342d3c8287183ea/roles/9e8771d0146c45e6afaf7915e215ac91` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:13.906877 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92ed72-7f3e-452b-abc4-ba5bd0da8116 None admin] RBAC: Authorizing `identity:create_grant(project_id=c872c66e5ba64576ab2eac8ce92c8b92, user_id=e4d728438a4f4a5ca342d3c8287183ea, role_id=9e8771d0146c45e6afaf7915e215ac91)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:13.908327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92ed72-7f3e-452b-abc4-ba5bd0da8116 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:13.931688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:13.942667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 150/292] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:04:13 2022] PUT /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92/users/e4d728438a4f4a5ca342d3c8287183ea/roles/9e8771d0146c45e6afaf7915e215ac91 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:04:13.983082 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915301d-e5ef-4d8e-bc16-0e959a69f842 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:14.043900 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993c228-b266-4d93-80f8-21c42624c5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:14.044810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993c228-b266-4d93-80f8-21c42624c5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:14.045310 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993c228-b266-4d93-80f8-21c42624c5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:14.045777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993c228-b266-4d93-80f8-21c42624c5c2 None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:14.050595 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993c228-b266-4d93-80f8-21c42624c5c2 None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:14.052463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993c228-b266-4d93-80f8-21c42624c5c2 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:14.055557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 143/293] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:13 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:14.061579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92ed72-7f3e-452b-abc4-ba5bd0da8116 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:14.192471 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92ed72-7f3e-452b-abc4-ba5bd0da8116 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:14.233533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b9ca7-0b5d-4594-bebf-4fa28d975fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:14.234154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b9ca7-0b5d-4594-bebf-4fa28d975fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:14.234435 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b9ca7-0b5d-4594-bebf-4fa28d975fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:14.234710 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b9ca7-0b5d-4594-bebf-4fa28d975fa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:14.235569 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b9ca7-0b5d-4594-bebf-4fa28d975fa7 None admin] RBAC: Adding query filter params (role.id=8fc75c351a0349e294deeed1ff391c4c, scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92, user.id=6ed69a73796d4f67b1a7458a294afa70) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:14.235881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b9ca7-0b5d-4594-bebf-4fa28d975fa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:14.237417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b9ca7-0b5d-4594-bebf-4fa28d975fa7 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:14.244729 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 151/294] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:14 2022] GET /identity/v3/role_assignments?user.id=6ed69a73796d4f67b1a7458a294afa70&scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92&role.id=8fc75c351a0349e294deeed1ff391c4c => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:14.825438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466c184-3b8c-464b-b699-7f204cb4d4c3 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:14.825815 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466c184-3b8c-464b-b699-7f204cb4d4c3 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:14.826003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f466c184-3b8c-464b-b699-7f204cb4d4c3 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:14.828085 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 144/295] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:14.833931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104ad57-168c-4244-ae3e-76285ba9fdc3 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:14.834396 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104ad57-168c-4244-ae3e-76285ba9fdc3 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:14.834583 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0104ad57-168c-4244-ae3e-76285ba9fdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:14.856254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-0104ad57-168c-4244-ae3e-76285ba9fdc3 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:14.857806 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-0104ad57-168c-4244-ae3e-76285ba9fdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:14.924867 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:14.938023 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 152/296] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:14.961014 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466c184-3b8c-464b-b699-7f204cb4d4c3 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:14.995468 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:15.062095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca57471-6a11-4662-984d-71c78212fa39 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:15.062379 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca57471-6a11-4662-984d-71c78212fa39 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:15.062734 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca57471-6a11-4662-984d-71c78212fa39 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:15.070673 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 153/297] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:04:15.072413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ca8a2-8ce5-4d9c-b176-a857957208f5 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:15.072724 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ca8a2-8ce5-4d9c-b176-a857957208f5 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:15.073065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ca8a2-8ce5-4d9c-b176-a857957208f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:15.077861 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f466c184-3b8c-464b-b699-7f204cb4d4c3 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:15.091665 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-fd5ca8a2-8ce5-4d9c-b176-a857957208f5 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:15.092899 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-fd5ca8a2-8ce5-4d9c-b176-a857957208f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:15.119213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3778b-a580-4984-a07a-8517df0704b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:15.120149 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3778b-a580-4984-a07a-8517df0704b8 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:15.120702 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3778b-a580-4984-a07a-8517df0704b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:15.121231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd3778b-a580-4984-a07a-8517df0704b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:15.135065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd3778b-a580-4984-a07a-8517df0704b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:15.142239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-0bd3778b-a580-4984-a07a-8517df0704b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:04:15.142239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:15.142239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:15.142239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:15.142239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:15.142239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:15.142239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:15.142239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:15.142239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:15.142239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:15.142239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:15.142239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:15.142239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:15.142239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:15.142239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:15.142239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:15.143287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:15.143287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:15.143287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:15.143287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:15.143287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:15.143287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:15.143287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:15.143287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:15.143287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:15.143287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:15.143287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:15.143287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:15.143287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:15.143287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:15.143287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:15.143287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:15.144267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:15.144267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:15.144267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:15.144267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:15.144267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:15.144267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:15.144267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:15.144267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:15.144267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:15.144267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:15.144267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:04:15.144267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:15.147006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 145/298] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:15.155897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3778b-a580-4984-a07a-8517df0704b8 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:15.171500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 154/299] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:04:15.195564 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ca8a2-8ce5-4d9c-b176-a857957208f5 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:15.253856 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:15.266164 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd3778b-a580-4984-a07a-8517df0704b8 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:15.304370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd91351-e2a7-4965-9e15-7d1445399c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:15.305360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd91351-e2a7-4965-9e15-7d1445399c55 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:15.305828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ca8a2-8ce5-4d9c-b176-a857957208f5 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:15.306100 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd91351-e2a7-4965-9e15-7d1445399c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:15.306557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd91351-e2a7-4965-9e15-7d1445399c55 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:15.307580 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd91351-e2a7-4965-9e15-7d1445399c55 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:15.308168 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd91351-e2a7-4965-9e15-7d1445399c55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:15.309778 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd91351-e2a7-4965-9e15-7d1445399c55 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:15.323719 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 146/300] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:15.325017 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd91351-e2a7-4965-9e15-7d1445399c55 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:15.330014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:15.342406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e7c97-da3d-4fb6-93a9-8cab73317e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:15.343045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e7c97-da3d-4fb6-93a9-8cab73317e47 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:15.343326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e7c97-da3d-4fb6-93a9-8cab73317e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:15.343602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e7c97-da3d-4fb6-93a9-8cab73317e47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:15.344881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e7c97-da3d-4fb6-93a9-8cab73317e47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:15.346367 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e7c97-da3d-4fb6-93a9-8cab73317e47 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:15.348649 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 155/301] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:15.354939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e7c97-da3d-4fb6-93a9-8cab73317e47 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:15.453445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd91351-e2a7-4965-9e15-7d1445399c55 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:15.471290 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e7c97-da3d-4fb6-93a9-8cab73317e47 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:15.489709 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde2a54-fb5a-41bd-8a34-bab8266a600a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:15.490600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde2a54-fb5a-41bd-8a34-bab8266a600a None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:15.491056 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde2a54-fb5a-41bd-8a34-bab8266a600a None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:15.491644 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde2a54-fb5a-41bd-8a34-bab8266a600a None admin] PATH_INFO: `/v3/users/95d922e15701432ca3706d257ad76756` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:15.506784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde2a54-fb5a-41bd-8a34-bab8266a600a None admin] RBAC: Authorizing `identity:get_user(user_id=95d922e15701432ca3706d257ad76756)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:15.507716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81be52a-2226-45c7-bdac-14c5380af6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:15.508276 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde2a54-fb5a-41bd-8a34-bab8266a600a None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:15.508421 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81be52a-2226-45c7-bdac-14c5380af6e4 None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:15.508593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81be52a-2226-45c7-bdac-14c5380af6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:15.508898 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81be52a-2226-45c7-bdac-14c5380af6e4 None admin] PATH_INFO: `/v3/groups` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:15.509780 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81be52a-2226-45c7-bdac-14c5380af6e4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:15.514879 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 147/302] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:15 2022] GET /identity/v3/users/95d922e15701432ca3706d257ad76756 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:15.515034 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81be52a-2226-45c7-bdac-14c5380af6e4 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:15.515121 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d4f0e-9ce6-465f-a2e6-a108c98a6f52 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:15.515121 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d4f0e-9ce6-465f-a2e6-a108c98a6f52 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:15.515121 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d4f0e-9ce6-465f-a2e6-a108c98a6f52 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:15.515121 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 148/303] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:15.516597 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d4f0e-9ce6-465f-a2e6-a108c98a6f52 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:15.530189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:15.532711 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 156/304] 10.222.0.45 () {60 vars in 1134 bytes} [Thu Aug 4 02:04:15 2022] POST /identity/v3/groups => generated 229 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:04:15.540845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f378a-0e98-45f7-822f-59b6ee896b87 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:15.540845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f378a-0e98-45f7-822f-59b6ee896b87 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:15.540845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f378a-0e98-45f7-822f-59b6ee896b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:15.570993 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-e78f378a-0e98-45f7-822f-59b6ee896b87 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:15.573151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e78f378a-0e98-45f7-822f-59b6ee896b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:15.640774 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d4f0e-9ce6-465f-a2e6-a108c98a6f52 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:15.652612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:15.689511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 157/305] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:15.694083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77a255-5755-4258-96f9-82a2af39cabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:15.694083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77a255-5755-4258-96f9-82a2af39cabb None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:15.694083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77a255-5755-4258-96f9-82a2af39cabb None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:15.694083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77a255-5755-4258-96f9-82a2af39cabb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:15.694401 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77a255-5755-4258-96f9-82a2af39cabb None admin] RBAC: Adding query filter params (role.id=9e8771d0146c45e6afaf7915e215ac91, user.id=95d922e15701432ca3706d257ad76756, scope.system=all) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:15.694401 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77a255-5755-4258-96f9-82a2af39cabb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:15.695135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77a255-5755-4258-96f9-82a2af39cabb None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:15.700460 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 149/306] 10.222.0.45 () {56 vars in 1308 bytes} [Thu Aug 4 02:04:15 2022] GET /identity/v3/role_assignments?user.id=95d922e15701432ca3706d257ad76756&scope.system=all&role.id=9e8771d0146c45e6afaf7915e215ac91 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:15.713469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f378a-0e98-45f7-822f-59b6ee896b87 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:15.830168 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f378a-0e98-45f7-822f-59b6ee896b87 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:15.868407 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3394ce-fb6b-4a7c-b3b6-10976d50fbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:15.869191 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3394ce-fb6b-4a7c-b3b6-10976d50fbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:15.869355 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3394ce-fb6b-4a7c-b3b6-10976d50fbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:15.869633 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3394ce-fb6b-4a7c-b3b6-10976d50fbd9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:15.880464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3394ce-fb6b-4a7c-b3b6-10976d50fbd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:15.885494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-1d3394ce-fb6b-4a7c-b3b6-10976d50fbd9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:15.885494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:15.885494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:15.885494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:15.885494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:15.885494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:15.885494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:15.885494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:15.885494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:15.885494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:15.885494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:15.885494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:15.885494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:15.885494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:15.885494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:15.885494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:15.886648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:15.886648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:15.886648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:15.886648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:15.886648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:15.886648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:15.886648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:15.886648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:15.886648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:15.886648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:15.886648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:15.886648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:15.886648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:15.886648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:15.886648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:15.886648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:15.887448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:15.887448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:15.887448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:15.887448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:15.887448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:15.887448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:15.887448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:15.887448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:15.887448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:15.887448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:15.887448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:15.887448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:15.887448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 158/307] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:15.894444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77a255-5755-4258-96f9-82a2af39cabb None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:15.980985 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:16.007146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77a255-5755-4258-96f9-82a2af39cabb None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:16.044425 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff442357-a0e4-40ca-af44-a3eb52be459d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:16.045081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff442357-a0e4-40ca-af44-a3eb52be459d None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:16.045360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff442357-a0e4-40ca-af44-a3eb52be459d None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:16.045628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff442357-a0e4-40ca-af44-a3eb52be459d None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:16.046347 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff442357-a0e4-40ca-af44-a3eb52be459d None admin] RBAC: Adding query filter params (name=member) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:16.046706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff442357-a0e4-40ca-af44-a3eb52be459d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:16.048142 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff442357-a0e4-40ca-af44-a3eb52be459d None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:16.057235 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 150/308] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:16.062931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3394ce-fb6b-4a7c-b3b6-10976d50fbd9 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:16.114296 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:16.180809 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3394ce-fb6b-4a7c-b3b6-10976d50fbd9 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:16.217261 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1931bdab-49b3-47dd-9b12-86622e5425ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:16.218247 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931bdab-49b3-47dd-9b12-86622e5425ae None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:16.218730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931bdab-49b3-47dd-9b12-86622e5425ae None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:16.219197 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1931bdab-49b3-47dd-9b12-86622e5425ae None admin] PATH_INFO: `/v3/users/e38c71c6201a427c8cb2a0d0dbcfa058` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:16.220608 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931bdab-49b3-47dd-9b12-86622e5425ae None admin] RBAC: Authorizing `identity:get_user(user_id=e38c71c6201a427c8cb2a0d0dbcfa058)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:16.222908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1931bdab-49b3-47dd-9b12-86622e5425ae None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:16.225112 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 159/309] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:16 2022] GET /identity/v3/users/e38c71c6201a427c8cb2a0d0dbcfa058 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:16.230929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff442357-a0e4-40ca-af44-a3eb52be459d None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:16.341086 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff442357-a0e4-40ca-af44-a3eb52be459d None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:16.366723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010909de-7aff-4816-b92f-98f28512c798 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:16.367037 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010909de-7aff-4816-b92f-98f28512c798 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:16.367277 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010909de-7aff-4816-b92f-98f28512c798 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:16.368935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 160/310] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:04:16.374059 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39da61-8957-47c9-8d9f-39a2364cb752 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:16.374354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39da61-8957-47c9-8d9f-39a2364cb752 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:16.374629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39da61-8957-47c9-8d9f-39a2364cb752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:16.376568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6630a6-2bec-415a-82f1-627045a232bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:16.377412 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6630a6-2bec-415a-82f1-627045a232bf None admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:16.377834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6630a6-2bec-415a-82f1-627045a232bf None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:16.378315 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6630a6-2bec-415a-82f1-627045a232bf None admin] PATH_INFO: `/v3/system/users/e38c71c6201a427c8cb2a0d0dbcfa058/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:16.379956 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6630a6-2bec-415a-82f1-627045a232bf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e38c71c6201a427c8cb2a0d0dbcfa058, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:16.381498 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6630a6-2bec-415a-82f1-627045a232bf None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:16.385362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:16.394860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-0e39da61-8957-47c9-8d9f-39a2364cb752 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:16.396279 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 151/311] 10.222.0.45 () {58 vars in 1334 bytes} [Thu Aug 4 02:04:16 2022] PUT /identity/v3/system/users/e38c71c6201a427c8cb2a0d0dbcfa058/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:04:16.396745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-0e39da61-8957-47c9-8d9f-39a2364cb752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:16.491323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 161/312] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:16.513850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6630a6-2bec-415a-82f1-627045a232bf None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:16.622688 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6630a6-2bec-415a-82f1-627045a232bf None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:16.666322 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e1580-d593-4bd3-88ef-9d96fe67d4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:16.666976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e1580-d593-4bd3-88ef-9d96fe67d4ae None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:16.667259 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e1580-d593-4bd3-88ef-9d96fe67d4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:16.667537 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e1580-d593-4bd3-88ef-9d96fe67d4ae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:16.678896 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e1580-d593-4bd3-88ef-9d96fe67d4ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:16.684350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-424e1580-d593-4bd3-88ef-9d96fe67d4ae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:04:16.684350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:16.684350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:16.684350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:16.684350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:16.684350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:16.684350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:16.684350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:16.684350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:16.684350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:16.684350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:16.684350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:16.684350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:16.684350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:16.684350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:16.684350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:16.685432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:16.685432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:16.685432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:16.685432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:16.685432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:16.685432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:16.685432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:16.685432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:16.685432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:16.685432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:16.685432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:16.685432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:16.685432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:16.685432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:16.685432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:16.685432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:16.686657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:16.686657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:16.686657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:16.686657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:16.686657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:16.686657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:16.686657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:16.686657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:16.686657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:16.686657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:16.686657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:04:16.686657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:16.686657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 152/313] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:16.692032 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39da61-8957-47c9-8d9f-39a2364cb752 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:16.732983 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:16.797275 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39da61-8957-47c9-8d9f-39a2364cb752 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:16.837894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95ef4d-c944-41c1-9340-9f6f2b09de3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:16.838537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95ef4d-c944-41c1-9340-9f6f2b09de3a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:16.838815 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95ef4d-c944-41c1-9340-9f6f2b09de3a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:16.839086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95ef4d-c944-41c1-9340-9f6f2b09de3a None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:16.839834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95ef4d-c944-41c1-9340-9f6f2b09de3a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:16.840139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95ef4d-c944-41c1-9340-9f6f2b09de3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:16.841620 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95ef4d-c944-41c1-9340-9f6f2b09de3a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:16.851790 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 162/314] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:16.857717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e1580-d593-4bd3-88ef-9d96fe67d4ae None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:16.920570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:16.965407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e1580-d593-4bd3-88ef-9d96fe67d4ae None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:16.999878 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3189ac-5a3b-4e72-85e9-d6e1319dc3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:17.000515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3189ac-5a3b-4e72-85e9-d6e1319dc3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:17.000824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3189ac-5a3b-4e72-85e9-d6e1319dc3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:17.001099 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3189ac-5a3b-4e72-85e9-d6e1319dc3d3 None admin] PATH_INFO: `/v3/users/e4d728438a4f4a5ca342d3c8287183ea` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:17.002269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3189ac-5a3b-4e72-85e9-d6e1319dc3d3 None admin] RBAC: Authorizing `identity:get_user(user_id=e4d728438a4f4a5ca342d3c8287183ea)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:17.003844 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3189ac-5a3b-4e72-85e9-d6e1319dc3d3 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:17.005490 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 153/315] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:16 2022] GET /identity/v3/users/e4d728438a4f4a5ca342d3c8287183ea => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:17.011178 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95ef4d-c944-41c1-9340-9f6f2b09de3a None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:17.113288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95ef4d-c944-41c1-9340-9f6f2b09de3a None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:17.114910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:17.149385 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83577665-e898-4d70-ae61-9288ed6f48b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:17.150007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83577665-e898-4d70-ae61-9288ed6f48b6 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:17.150285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83577665-e898-4d70-ae61-9288ed6f48b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:17.150555 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83577665-e898-4d70-ae61-9288ed6f48b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:17.221268 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83577665-e898-4d70-ae61-9288ed6f48b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:17.222799 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83577665-e898-4d70-ae61-9288ed6f48b6 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:17.322711 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:17.331762 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 163/316] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:17.333206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3189ac-5a3b-4e72-85e9-d6e1319dc3d3 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:17.437200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3189ac-5a3b-4e72-85e9-d6e1319dc3d3 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:17.469652 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:17.479424 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253d6f79-edd3-4286-9714-307094284573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:17.479998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d6f79-edd3-4286-9714-307094284573 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:17.480280 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d6f79-edd3-4286-9714-307094284573 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:17.480559 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253d6f79-edd3-4286-9714-307094284573 None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:17.481767 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d6f79-edd3-4286-9714-307094284573 None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:17.483236 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253d6f79-edd3-4286-9714-307094284573 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:17.485246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 154/317] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:17 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:17.491440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83577665-e898-4d70-ae61-9288ed6f48b6 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:17.594931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83577665-e898-4d70-ae61-9288ed6f48b6 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:17.631330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ac3831-9e0f-4345-a16f-e93236341564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:17.632320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac3831-9e0f-4345-a16f-e93236341564 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:17.632833 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac3831-9e0f-4345-a16f-e93236341564 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:17.633304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ac3831-9e0f-4345-a16f-e93236341564 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:17.634321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac3831-9e0f-4345-a16f-e93236341564 None admin] RBAC: Adding query filter params (role.id=9e8771d0146c45e6afaf7915e215ac91, scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92, user.id=e4d728438a4f4a5ca342d3c8287183ea) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:17.634831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac3831-9e0f-4345-a16f-e93236341564 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:17.636519 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ac3831-9e0f-4345-a16f-e93236341564 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:17.643541 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 164/318] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:17 2022] GET /identity/v3/role_assignments?user.id=e4d728438a4f4a5ca342d3c8287183ea&scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92&role.id=9e8771d0146c45e6afaf7915e215ac91 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:17.841600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99862a7a-5b1a-4e6f-876c-d0047d6cebe7 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:17.842025 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99862a7a-5b1a-4e6f-876c-d0047d6cebe7 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:17.842323 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99862a7a-5b1a-4e6f-876c-d0047d6cebe7 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:17.843689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 155/319] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:17.844095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7f386-d4db-4f51-b593-5a2bfd79b95a None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:17.844693 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7f386-d4db-4f51-b593-5a2bfd79b95a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:17.845169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7f386-d4db-4f51-b593-5a2bfd79b95a None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:17.846719 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 165/320] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:17.849451 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbe037-c99f-45b3-ac94-15a905c57bed None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:17.849948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbe037-c99f-45b3-ac94-15a905c57bed None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:17.850459 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbe037-c99f-45b3-ac94-15a905c57bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:17.852984 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223dea1-17f1-4cc6-aaf0-edb1a8e10142 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:17.853264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223dea1-17f1-4cc6-aaf0-edb1a8e10142 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:17.853538 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4223dea1-17f1-4cc6-aaf0-edb1a8e10142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:17.871591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-d3fbe037-c99f-45b3-ac94-15a905c57bed None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:17.873020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d3fbe037-c99f-45b3-ac94-15a905c57bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:17.881071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-4223dea1-17f1-4cc6-aaf0-edb1a8e10142 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:17.882264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4223dea1-17f1-4cc6-aaf0-edb1a8e10142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:17.951643 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:17.960666 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 156/321] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:17.961403 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 166/322] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:04:17.980923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbe037-c99f-45b3-ac94-15a905c57bed None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:17.982771 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223dea1-17f1-4cc6-aaf0-edb1a8e10142 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:18.090345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbe037-c99f-45b3-ac94-15a905c57bed None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:18.091780 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4223dea1-17f1-4cc6-aaf0-edb1a8e10142 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:18.120420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:18.125933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660cb35-82b6-43b2-b33b-5d51f91f66ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:18.126576 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660cb35-82b6-43b2-b33b-5d51f91f66ae None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:18.126859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660cb35-82b6-43b2-b33b-5d51f91f66ae None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:18.127129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660cb35-82b6-43b2-b33b-5d51f91f66ae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:18.132090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1cf12a-e699-4f1d-85d6-9c4c696ffff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:18.132763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cf12a-e699-4f1d-85d6-9c4c696ffff0 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:18.133048 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cf12a-e699-4f1d-85d6-9c4c696ffff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:18.133321 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1cf12a-e699-4f1d-85d6-9c4c696ffff0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:18.134631 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1cf12a-e699-4f1d-85d6-9c4c696ffff0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:18.136107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1cf12a-e699-4f1d-85d6-9c4c696ffff0 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:18.137744 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 157/323] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:18.144755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1cf12a-e699-4f1d-85d6-9c4c696ffff0 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:18.149230 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660cb35-82b6-43b2-b33b-5d51f91f66ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:18.154243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-7660cb35-82b6-43b2-b33b-5d51f91f66ae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:04:18.154243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:18.154243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:18.154243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:18.154243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:18.154243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:18.154243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:18.154243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:18.154243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:18.154243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:18.154243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:18.154243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:18.154243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:18.154243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:18.154243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:18.154243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:18.155156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:18.155156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:18.155156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:18.155156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:18.155156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:18.155156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:18.155156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:18.155156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:18.155156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:18.155156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:18.155156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:18.155156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:18.155156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:18.155156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:18.155156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:18.155156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:18.157243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:18.157243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:18.157243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:18.157243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:18.157243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:18.157243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:18.157243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:18.157243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:18.157243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:18.157243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:18.157243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:04:18.157243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:18.157243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 167/324] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:04:18.161098 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660cb35-82b6-43b2-b33b-5d51f91f66ae None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:18.218111 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:18.261633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1cf12a-e699-4f1d-85d6-9c4c696ffff0 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:18.266547 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660cb35-82b6-43b2-b33b-5d51f91f66ae None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:18.300426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099a820-b980-49e3-8d2a-b096dc81c2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:18.301447 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099a820-b980-49e3-8d2a-b096dc81c2d6 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:18.301563 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41794762-eec3-4ed0-9b70-b3b56378a4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:18.301978 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099a820-b980-49e3-8d2a-b096dc81c2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:18.302132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41794762-eec3-4ed0-9b70-b3b56378a4ee None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:18.302418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41794762-eec3-4ed0-9b70-b3b56378a4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:18.302696 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41794762-eec3-4ed0-9b70-b3b56378a4ee None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:18.302799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d099a820-b980-49e3-8d2a-b096dc81c2d6 None admin] PATH_INFO: `/v3/groups` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:18.303414 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41794762-eec3-4ed0-9b70-b3b56378a4ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:18.303721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41794762-eec3-4ed0-9b70-b3b56378a4ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:18.303861 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099a820-b980-49e3-8d2a-b096dc81c2d6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:18.305144 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41794762-eec3-4ed0-9b70-b3b56378a4ee None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:18.305595 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d099a820-b980-49e3-8d2a-b096dc81c2d6 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:18.314654 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 168/325] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:18.320012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41794762-eec3-4ed0-9b70-b3b56378a4ee None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:18.330445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 158/326] 10.222.0.45 () {60 vars in 1134 bytes} [Thu Aug 4 02:04:18 2022] POST /identity/v3/groups => generated 226 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:04:18.430614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41794762-eec3-4ed0-9b70-b3b56378a4ee None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:18.451117 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:18.470594 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b830e3-8a1a-401d-b4df-b8c992a42590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:18.471251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b830e3-8a1a-401d-b4df-b8c992a42590 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:18.471534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b830e3-8a1a-401d-b4df-b8c992a42590 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:18.471856 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b830e3-8a1a-401d-b4df-b8c992a42590 None admin] PATH_INFO: `/v3/users/95d922e15701432ca3706d257ad76756` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:18.473108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b830e3-8a1a-401d-b4df-b8c992a42590 None admin] RBAC: Authorizing `identity:get_user(user_id=95d922e15701432ca3706d257ad76756)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:18.474576 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b830e3-8a1a-401d-b4df-b8c992a42590 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:18.476371 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 169/327] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:18 2022] GET /identity/v3/users/95d922e15701432ca3706d257ad76756 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:18.482104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099a820-b980-49e3-8d2a-b096dc81c2d6 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:18.546867 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:18.598437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d099a820-b980-49e3-8d2a-b096dc81c2d6 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:18.635819 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ae260-9a8a-427a-bf82-b1ccf297365e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:18.636492 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ae260-9a8a-427a-bf82-b1ccf297365e None admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:18.636810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ae260-9a8a-427a-bf82-b1ccf297365e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:18.637088 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ae260-9a8a-427a-bf82-b1ccf297365e None admin] PATH_INFO: `/v3/system/users/95d922e15701432ca3706d257ad76756/roles/9e8771d0146c45e6afaf7915e215ac91` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:18.638596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ae260-9a8a-427a-bf82-b1ccf297365e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=95d922e15701432ca3706d257ad76756, role_id=9e8771d0146c45e6afaf7915e215ac91)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:18.639992 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ae260-9a8a-427a-bf82-b1ccf297365e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:18.642419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176d2ba-7dde-45cd-b8bb-2dc61e3003f6 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:18.642702 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176d2ba-7dde-45cd-b8bb-2dc61e3003f6 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:18.642975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176d2ba-7dde-45cd-b8bb-2dc61e3003f6 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:18.646758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 170/328] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:18.650295 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787a92a-74d6-4e13-a200-213b7a949ddf None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:18.650709 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787a92a-74d6-4e13-a200-213b7a949ddf None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:18.651079 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787a92a-74d6-4e13-a200-213b7a949ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:18.655730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 159/329] 10.222.0.45 () {58 vars in 1334 bytes} [Thu Aug 4 02:04:18 2022] PUT /identity/v3/system/users/95d922e15701432ca3706d257ad76756/roles/9e8771d0146c45e6afaf7915e215ac91 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:04:18.674496 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-e787a92a-74d6-4e13-a200-213b7a949ddf None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:18.676355 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e787a92a-74d6-4e13-a200-213b7a949ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:18.767721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 171/330] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:18.791944 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ae260-9a8a-427a-bf82-b1ccf297365e None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:18.912275 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ae260-9a8a-427a-bf82-b1ccf297365e None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:18.933792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:18.960600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a553b-3f5b-4667-891d-f67c05bf57a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:18.961649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a553b-3f5b-4667-891d-f67c05bf57a2 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:18.962132 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a553b-3f5b-4667-891d-f67c05bf57a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:18.962629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394a553b-3f5b-4667-891d-f67c05bf57a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:18.977857 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394a553b-3f5b-4667-891d-f67c05bf57a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:18.983241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-394a553b-3f5b-4667-891d-f67c05bf57a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:18.983241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:18.983241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:18.983241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:18.983241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:18.983241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:18.983241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:18.983241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:18.983241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:18.983241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:18.983241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:18.983241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:18.983241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:18.983241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:18.983241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:18.983241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:18.984146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:18.984146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:18.984146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:18.984146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:18.984146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:18.984146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:18.984146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:18.984146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:18.984146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:18.984146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:18.984146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:18.984146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:18.984146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:18.984146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:18.984146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:18.984146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:18.984999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:18.984999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:18.984999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:18.984999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:18.984999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:18.984999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:18.984999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:18.984999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:18.984999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:18.984999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:18.984999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:18.984999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:18.985596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 160/331] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:18.992087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787a92a-74d6-4e13-a200-213b7a949ddf None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:19.121764 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787a92a-74d6-4e13-a200-213b7a949ddf None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:19.137896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:19.163224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f3b7a-230e-49d5-8bc8-8ba8f08c502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:19.163866 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3b7a-230e-49d5-8bc8-8ba8f08c502e None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:19.164194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3b7a-230e-49d5-8bc8-8ba8f08c502e None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:19.164419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292f3b7a-230e-49d5-8bc8-8ba8f08c502e None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:19.165185 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f3b7a-230e-49d5-8bc8-8ba8f08c502e None admin] RBAC: Adding query filter params (name=member) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:19.165487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f3b7a-230e-49d5-8bc8-8ba8f08c502e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:19.166876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292f3b7a-230e-49d5-8bc8-8ba8f08c502e None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:19.176368 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 172/332] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:19.181919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a553b-3f5b-4667-891d-f67c05bf57a2 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:19.292115 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394a553b-3f5b-4667-891d-f67c05bf57a2 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:19.332858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea367948-d718-4355-a696-c72d635c6b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:19.333560 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea367948-d718-4355-a696-c72d635c6b32 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:19.333840 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea367948-d718-4355-a696-c72d635c6b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:19.334115 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea367948-d718-4355-a696-c72d635c6b32 None admin] PATH_INFO: `/v3/users/e38c71c6201a427c8cb2a0d0dbcfa058` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:19.335357 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea367948-d718-4355-a696-c72d635c6b32 None admin] RBAC: Authorizing `identity:get_user(user_id=e38c71c6201a427c8cb2a0d0dbcfa058)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:19.336834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea367948-d718-4355-a696-c72d635c6b32 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:19.338606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 161/333] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:19 2022] GET /identity/v3/users/e38c71c6201a427c8cb2a0d0dbcfa058 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:19.344292 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f3b7a-230e-49d5-8bc8-8ba8f08c502e None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:19.469209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292f3b7a-230e-49d5-8bc8-8ba8f08c502e None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:19.512148 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a5db50-fe4a-485e-822b-3b7af5d9c903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:19.513204 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5db50-fe4a-485e-822b-3b7af5d9c903 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:19.513689 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5db50-fe4a-485e-822b-3b7af5d9c903 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:19.514181 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5db50-fe4a-485e-822b-3b7af5d9c903 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:19.515175 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5db50-fe4a-485e-822b-3b7af5d9c903 None admin] RBAC: Adding query filter params (role.id=8fc75c351a0349e294deeed1ff391c4c, user.id=e38c71c6201a427c8cb2a0d0dbcfa058, scope.system=all) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:19.515685 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5db50-fe4a-485e-822b-3b7af5d9c903 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:19.517392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a5db50-fe4a-485e-822b-3b7af5d9c903 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:19.535470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 173/334] 10.222.0.45 () {56 vars in 1308 bytes} [Thu Aug 4 02:04:19 2022] GET /identity/v3/role_assignments?user.id=e38c71c6201a427c8cb2a0d0dbcfa058&scope.system=all&role.id=8fc75c351a0349e294deeed1ff391c4c => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:20.749567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2424a8-eb57-4666-b3ce-912fa787a854 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:20.750521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2424a8-eb57-4666-b3ce-912fa787a854 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:20.750997 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2424a8-eb57-4666-b3ce-912fa787a854 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:20.752872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 162/335] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:20.758609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bcfcb-c9a7-40d7-abde-9eed7ae1318c None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:20.759041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bcfcb-c9a7-40d7-abde-9eed7ae1318c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:20.759160 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bcfcb-c9a7-40d7-abde-9eed7ae1318c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:20.784446 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-bf7bcfcb-c9a7-40d7-abde-9eed7ae1318c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:20.785923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-bf7bcfcb-c9a7-40d7-abde-9eed7ae1318c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:20.820371 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:20.883397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 174/336] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:20.927113 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2424a8-eb57-4666-b3ce-912fa787a854 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:21.004525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:21.045525 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ef87a-5305-4799-8d54-13f36f807a4b None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:21.046091 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ef87a-5305-4799-8d54-13f36f807a4b None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:21.046570 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ef87a-5305-4799-8d54-13f36f807a4b None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:21.048096 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 175/337] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:04:21.054222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097967aa-1982-415e-b643-3a8859efb259 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:21.054863 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097967aa-1982-415e-b643-3a8859efb259 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:21.055207 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097967aa-1982-415e-b643-3a8859efb259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:21.085971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2424a8-eb57-4666-b3ce-912fa787a854 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:21.086241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-097967aa-1982-415e-b643-3a8859efb259 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:21.087715 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-097967aa-1982-415e-b643-3a8859efb259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:21.153334 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd95d31-d68a-4423-b071-a1618adf4c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:21.153984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd95d31-d68a-4423-b071-a1618adf4c0d None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:21.154261 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd95d31-d68a-4423-b071-a1618adf4c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:21.154530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd95d31-d68a-4423-b071-a1618adf4c0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:21.181622 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd95d31-d68a-4423-b071-a1618adf4c0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:21.188123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-fdd95d31-d68a-4423-b071-a1618adf4c0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:21.188123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:21.188123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:21.188123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:21.188123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:21.188123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:21.188123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:21.188123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:21.188123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:21.188123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:21.188123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:21.188123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:21.188123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:21.188123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:21.188123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:21.188123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:21.189051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:21.189051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:21.189051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:21.189051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:21.189051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:21.189051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:21.189051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:21.189051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:21.189051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:21.189051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:21.189051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:21.189051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:21.189051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:21.189051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:21.189051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:21.189051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:21.189850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:21.189850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:21.189850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:21.189850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:21.189850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:21.189850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:21.189850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:21.189850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:21.189850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:21.189850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:21.189850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:21.189850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:21.193746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 163/338] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:21.201607 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd95d31-d68a-4423-b071-a1618adf4c0d None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:21.247646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 176/339] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:21.294336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097967aa-1982-415e-b643-3a8859efb259 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:21.388528 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd95d31-d68a-4423-b071-a1618adf4c0d None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:21.397214 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:21.433602 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439543b-aa99-4b29-b8ae-779f626d4a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:21.434367 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439543b-aa99-4b29-b8ae-779f626d4a0f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:21.434652 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439543b-aa99-4b29-b8ae-779f626d4a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:21.435247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439543b-aa99-4b29-b8ae-779f626d4a0f None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:21.436248 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439543b-aa99-4b29-b8ae-779f626d4a0f None admin] RBAC: Adding query filter params (name=member) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:21.436548 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439543b-aa99-4b29-b8ae-779f626d4a0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:21.438426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439543b-aa99-4b29-b8ae-779f626d4a0f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:21.444592 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:21.455560 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 164/340] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:21.457235 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2dc54-931e-4f61-a906-b8c27836e7b9 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:21.458267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2dc54-931e-4f61-a906-b8c27836e7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:21.458550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2dc54-931e-4f61-a906-b8c27836e7b9 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:21.459910 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 165/341] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:21.462285 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2dc54-931e-4f61-a906-b8c27836e7b9 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:21.462408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097967aa-1982-415e-b643-3a8859efb259 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:21.505274 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d72ad9-2697-4524-8555-437236d0131f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:21.506054 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d72ad9-2697-4524-8555-437236d0131f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:21.506430 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d72ad9-2697-4524-8555-437236d0131f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:21.507800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d72ad9-2697-4524-8555-437236d0131f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:21.520102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d72ad9-2697-4524-8555-437236d0131f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:21.527405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-98d72ad9-2697-4524-8555-437236d0131f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:21.527405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:21.527405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:21.527405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:21.527405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:21.527405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:21.527405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:21.527405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:21.527405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:21.527405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:21.527405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:21.527405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:21.527405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:21.527405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:21.527405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:21.527405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:21.528343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:21.528343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:21.528343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:21.528343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:21.528343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:21.528343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:21.528343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:21.528343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:21.528343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:21.528343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:21.528343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:21.528343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:21.528343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:21.528343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:21.528343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:21.528343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:21.529171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:21.529171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:21.529171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:21.529171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:21.529171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:21.529171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:21.529171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:21.529171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:21.529171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:21.529171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:21.529171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:21.529171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:21.530402 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 177/342] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:04:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:04:21.530727 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d70f68-b46a-4157-b57d-26ce40c25aff None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:21.531014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d70f68-b46a-4157-b57d-26ce40c25aff None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:21.531286 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d70f68-b46a-4157-b57d-26ce40c25aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:21.566517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-70d70f68-b46a-4157-b57d-26ce40c25aff None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:21.567708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-70d70f68-b46a-4157-b57d-26ce40c25aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:21.632582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f2dc54-931e-4f61-a906-b8c27836e7b9 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:21.672034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee75b7-f13d-4fda-bc98-d17683eb75d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:21.672739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee75b7-f13d-4fda-bc98-d17683eb75d8 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:21.673176 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee75b7-f13d-4fda-bc98-d17683eb75d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:21.673623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee75b7-f13d-4fda-bc98-d17683eb75d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:21.714342 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:21.717272 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 178/343] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:21.718162 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d70f68-b46a-4157-b57d-26ce40c25aff None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:21.754831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee75b7-f13d-4fda-bc98-d17683eb75d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:21.756972 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee75b7-f13d-4fda-bc98-d17683eb75d8 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:21.810526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:21.830702 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d70f68-b46a-4157-b57d-26ce40c25aff None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:21.865416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74e630-6dd9-4a75-a51a-2ff6b250a4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:21.866014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74e630-6dd9-4a75-a51a-2ff6b250a4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:21.866297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74e630-6dd9-4a75-a51a-2ff6b250a4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:21.866569 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74e630-6dd9-4a75-a51a-2ff6b250a4d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:21.867292 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74e630-6dd9-4a75-a51a-2ff6b250a4d5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:21.867596 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74e630-6dd9-4a75-a51a-2ff6b250a4d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:21.870912 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74e630-6dd9-4a75-a51a-2ff6b250a4d5 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:21.871079 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 166/344] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:21.872059 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee75b7-f13d-4fda-bc98-d17683eb75d8 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:21.884971 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 179/345] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:21.886243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74e630-6dd9-4a75-a51a-2ff6b250a4d5 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:21.948211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:21.988845 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee75b7-f13d-4fda-bc98-d17683eb75d8 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:21.995194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74e630-6dd9-4a75-a51a-2ff6b250a4d5 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:22.030483 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fec88-1aea-48c7-bdbe-d41ba651088b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:22.031469 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fec88-1aea-48c7-bdbe-d41ba651088b None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:22.031964 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fec88-1aea-48c7-bdbe-d41ba651088b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:22.032441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fec88-1aea-48c7-bdbe-d41ba651088b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:22.032607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb7542-4aa6-4568-8e3d-74cab4222996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:22.033242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb7542-4aa6-4568-8e3d-74cab4222996 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:22.033538 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb7542-4aa6-4568-8e3d-74cab4222996 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:22.033960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb7542-4aa6-4568-8e3d-74cab4222996 None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:22.035715 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb7542-4aa6-4568-8e3d-74cab4222996 None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:22.037605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb7542-4aa6-4568-8e3d-74cab4222996 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:22.039650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 180/346] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:21 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:22.040427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb7542-4aa6-4568-8e3d-74cab4222996 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:22.046707 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fec88-1aea-48c7-bdbe-d41ba651088b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:22.052581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-b17fec88-1aea-48c7-bdbe-d41ba651088b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:04:22.052581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:22.052581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:22.052581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:22.052581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:22.052581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:22.052581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:22.052581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:22.052581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:22.052581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:22.052581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:22.052581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:22.052581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:22.052581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:22.052581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:22.052581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:22.053514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:22.053514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:22.053514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:22.053514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:22.053514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:22.053514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:22.053514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:22.053514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:22.053514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:22.053514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:22.053514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:22.053514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:22.053514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:22.053514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:22.053514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:22.053514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:22.054397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:22.054397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:22.054397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:22.054397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:22.054397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:22.054397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:22.054397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:22.054397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:22.054397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:22.054397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:22.054397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 04 02:04:22.054397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:22.056940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 167/347] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:22.058070 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fec88-1aea-48c7-bdbe-d41ba651088b None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:22.061853 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:22.159075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb7542-4aa6-4568-8e3d-74cab4222996 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:22.173081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fec88-1aea-48c7-bdbe-d41ba651088b None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:22.198592 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8603df6-4994-4c30-b80e-2e18603eba92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:22.199802 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8603df6-4994-4c30-b80e-2e18603eba92 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:22.200329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8603df6-4994-4c30-b80e-2e18603eba92 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:22.200575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8603df6-4994-4c30-b80e-2e18603eba92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:22.203816 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:22.213021 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49771aa6-2ad0-4dc7-a77b-0315b4df1d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:22.213805 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49771aa6-2ad0-4dc7-a77b-0315b4df1d54 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:22.214175 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49771aa6-2ad0-4dc7-a77b-0315b4df1d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:22.214545 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49771aa6-2ad0-4dc7-a77b-0315b4df1d54 None admin] PATH_INFO: `/v3/groups/a51f0767088e4c889d3c33d8675952ec` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:22.221664 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49771aa6-2ad0-4dc7-a77b-0315b4df1d54 None admin] RBAC: Authorizing `identity:get_group(group_id=a51f0767088e4c889d3c33d8675952ec)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:22.223932 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49771aa6-2ad0-4dc7-a77b-0315b4df1d54 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:22.226367 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 168/348] 10.222.0.45 () {56 vars in 1180 bytes} [Thu Aug 4 02:04:22 2022] GET /identity/v3/groups/a51f0767088e4c889d3c33d8675952ec => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:22.227332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49771aa6-2ad0-4dc7-a77b-0315b4df1d54 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:22.291775 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8603df6-4994-4c30-b80e-2e18603eba92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:22.293710 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8603df6-4994-4c30-b80e-2e18603eba92 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:22.309735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:22.339791 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49771aa6-2ad0-4dc7-a77b-0315b4df1d54 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:22.376072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714ce9f-23e7-4518-9ec8-cd4937a6528c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:22.377074 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714ce9f-23e7-4518-9ec8-cd4937a6528c None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:22.377550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714ce9f-23e7-4518-9ec8-cd4937a6528c None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:22.378696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d714ce9f-23e7-4518-9ec8-cd4937a6528c None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:22.379648 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714ce9f-23e7-4518-9ec8-cd4937a6528c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:22.380236 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714ce9f-23e7-4518-9ec8-cd4937a6528c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:22.381843 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d714ce9f-23e7-4518-9ec8-cd4937a6528c None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:22.391561 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 169/349] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:22.392745 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714ce9f-23e7-4518-9ec8-cd4937a6528c None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:22.405238 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 181/350] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:22.405885 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8603df6-4994-4c30-b80e-2e18603eba92 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:22.458471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:22.509692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d714ce9f-23e7-4518-9ec8-cd4937a6528c None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:22.519934 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8603df6-4994-4c30-b80e-2e18603eba92 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:22.535714 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:22.549255 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eff64b-d088-4f1c-8c07-89ef1983317b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:22.550404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eff64b-d088-4f1c-8c07-89ef1983317b None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:22.550925 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eff64b-d088-4f1c-8c07-89ef1983317b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:22.551435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eff64b-d088-4f1c-8c07-89ef1983317b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:22.552484 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eff64b-d088-4f1c-8c07-89ef1983317b None admin] RBAC: Adding query filter params (group.id=a51f0767088e4c889d3c33d8675952ec, role.id=8fc75c351a0349e294deeed1ff391c4c, scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:22.553021 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eff64b-d088-4f1c-8c07-89ef1983317b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:22.554793 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eff64b-d088-4f1c-8c07-89ef1983317b None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:22.559056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81ff70-5aaa-4601-98c9-55afd6dc2bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:22.559661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81ff70-5aaa-4601-98c9-55afd6dc2bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:22.559945 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81ff70-5aaa-4601-98c9-55afd6dc2bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:22.560217 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae81ff70-5aaa-4601-98c9-55afd6dc2bc5 None admin] PATH_INFO: `/v3/users/95d922e15701432ca3706d257ad76756` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:22.561675 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae81ff70-5aaa-4601-98c9-55afd6dc2bc5 None admin] RBAC: Authorizing `identity:get_user(user_id=95d922e15701432ca3706d257ad76756)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:22.562283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 170/351] 10.222.0.45 () {56 vars in 1376 bytes} [Thu Aug 4 02:04:22 2022] GET /identity/v3/role_assignments?group.id=a51f0767088e4c889d3c33d8675952ec&scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef&role.id=8fc75c351a0349e294deeed1ff391c4c => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:22.563111 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae81ff70-5aaa-4601-98c9-55afd6dc2bc5 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:22.563515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eff64b-d088-4f1c-8c07-89ef1983317b None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:22.565008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 182/352] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:04:22 2022] GET /identity/v3/users/95d922e15701432ca3706d257ad76756 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:22.571191 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81ff70-5aaa-4601-98c9-55afd6dc2bc5 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:22.675447 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eff64b-d088-4f1c-8c07-89ef1983317b None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:22.696218 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:22.715928 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320be771-fc54-4e2c-8be4-fea281a50dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:22.717046 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320be771-fc54-4e2c-8be4-fea281a50dba None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:22.717522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320be771-fc54-4e2c-8be4-fea281a50dba None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:22.717986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320be771-fc54-4e2c-8be4-fea281a50dba None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:22.720119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320be771-fc54-4e2c-8be4-fea281a50dba None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:22.722062 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320be771-fc54-4e2c-8be4-fea281a50dba None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:22.724024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 171/353] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:22 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:22.726639 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae81ff70-5aaa-4601-98c9-55afd6dc2bc5 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:22.730020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320be771-fc54-4e2c-8be4-fea281a50dba None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:22.790041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714097c1-9f1f-46b5-8a26-07ce53cbb395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:22.790420 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714097c1-9f1f-46b5-8a26-07ce53cbb395 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:22.790639 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714097c1-9f1f-46b5-8a26-07ce53cbb395 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:22.790913 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714097c1-9f1f-46b5-8a26-07ce53cbb395 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:22.791731 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714097c1-9f1f-46b5-8a26-07ce53cbb395 None admin] RBAC: Adding query filter params (role.id=9e8771d0146c45e6afaf7915e215ac91, user.id=95d922e15701432ca3706d257ad76756, scope.system=all) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:22.792052 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714097c1-9f1f-46b5-8a26-07ce53cbb395 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:22.793650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714097c1-9f1f-46b5-8a26-07ce53cbb395 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:22.807725 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 183/354] 10.222.0.45 () {56 vars in 1308 bytes} [Thu Aug 4 02:04:22 2022] GET /identity/v3/role_assignments?user.id=95d922e15701432ca3706d257ad76756&scope.system=all&role.id=9e8771d0146c45e6afaf7915e215ac91 => generated 510 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:22.808996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad44bf-5e57-4fde-80b4-04cc543c91c7 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:22.809228 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad44bf-5e57-4fde-80b4-04cc543c91c7 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:22.809503 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad44bf-5e57-4fde-80b4-04cc543c91c7 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:22.816241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 184/355] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:22.818104 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b3765-afb2-4ca2-8e26-65658d35debd None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:22.818387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b3765-afb2-4ca2-8e26-65658d35debd None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:22.819213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3b3765-afb2-4ca2-8e26-65658d35debd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:22.838461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-da3b3765-afb2-4ca2-8e26-65658d35debd None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:22.839698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-da3b3765-afb2-4ca2-8e26-65658d35debd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:22.858809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320be771-fc54-4e2c-8be4-fea281a50dba None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:22.886262 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:22.904836 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14decfb-596b-4a19-93f9-83543cce9a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:22.905876 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14decfb-596b-4a19-93f9-83543cce9a47 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:22.906223 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14decfb-596b-4a19-93f9-83543cce9a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:22.906623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14decfb-596b-4a19-93f9-83543cce9a47 None admin] PATH_INFO: `/v3/groups/a51f0767088e4c889d3c33d8675952ec` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:22.908172 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14decfb-596b-4a19-93f9-83543cce9a47 None admin] RBAC: Authorizing `identity:get_group(group_id=a51f0767088e4c889d3c33d8675952ec)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:22.910161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14decfb-596b-4a19-93f9-83543cce9a47 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:22.912693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 172/356] 10.222.0.45 () {56 vars in 1180 bytes} [Thu Aug 4 02:04:22 2022] GET /identity/v3/groups/a51f0767088e4c889d3c33d8675952ec => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:22.918312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14decfb-596b-4a19-93f9-83543cce9a47 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:22.929841 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 185/357] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:04:22.998317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:23.021700 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b3765-afb2-4ca2-8e26-65658d35debd None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:23.053606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14decfb-596b-4a19-93f9-83543cce9a47 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:23.097146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c0c67-e59b-4605-8c30-efd9cd451be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:23.097712 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c0c67-e59b-4605-8c30-efd9cd451be5 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:23.098054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c0c67-e59b-4605-8c30-efd9cd451be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:23.098334 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c0c67-e59b-4605-8c30-efd9cd451be5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:23.099139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c0c67-e59b-4605-8c30-efd9cd451be5 None admin] RBAC: Adding query filter params (group.id=a51f0767088e4c889d3c33d8675952ec, role.id=80e5528d7c8742f78e29e64c1216176c, scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:23.099448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c0c67-e59b-4605-8c30-efd9cd451be5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:23.100930 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c0c67-e59b-4605-8c30-efd9cd451be5 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:23.108954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 173/358] 10.222.0.45 () {56 vars in 1376 bytes} [Thu Aug 4 02:04:22 2022] GET /identity/v3/role_assignments?group.id=a51f0767088e4c889d3c33d8675952ec&scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef&role.id=80e5528d7c8742f78e29e64c1216176c => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:23.110700 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6a377-d944-4ec8-8f6d-c7bfb18a0777 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:23.110809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6a377-d944-4ec8-8f6d-c7bfb18a0777 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:23.111078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6a377-d944-4ec8-8f6d-c7bfb18a0777 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:23.113095 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 174/359] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:23.119705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278bbfd-e0f3-4d59-a4e9-f291a66c9e58 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:23.120012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278bbfd-e0f3-4d59-a4e9-f291a66c9e58 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:23.120252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278bbfd-e0f3-4d59-a4e9-f291a66c9e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:23.146309 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-5278bbfd-e0f3-4d59-a4e9-f291a66c9e58 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:23.147868 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-5278bbfd-e0f3-4d59-a4e9-f291a66c9e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:23.185001 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3b3765-afb2-4ca2-8e26-65658d35debd None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:23.235686 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb64d9-46b7-46ec-9f5c-82d1c0e5e6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:23.235686 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb64d9-46b7-46ec-9f5c-82d1c0e5e6b9 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:23.235686 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb64d9-46b7-46ec-9f5c-82d1c0e5e6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:23.235686 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eb64d9-46b7-46ec-9f5c-82d1c0e5e6b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:23.251763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 175/360] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:04:23.261424 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69eb64d9-46b7-46ec-9f5c-82d1c0e5e6b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:23.267045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-69eb64d9-46b7-46ec-9f5c-82d1c0e5e6b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:23.267045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:23.267045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:23.267045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:23.267045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:23.267045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:23.267045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:23.267045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:23.267045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:23.267045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:23.267045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:23.267045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:23.267045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:23.267045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:23.267045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:23.267045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:23.267950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:23.267950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:23.267950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:23.267950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:23.267950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:23.267950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:23.267950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:23.267950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:23.267950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:23.267950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:23.267950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:23.267950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:23.267950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:23.267950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:23.267950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:23.267950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:23.269433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:23.269433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:23.269433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:23.269433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:23.269433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:23.269433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:23.269433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:23.269433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:23.269433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:23.269433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:23.269433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:23.269433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:23.272406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 186/361] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:04:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:23.278224 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278bbfd-e0f3-4d59-a4e9-f291a66c9e58 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:23.286117 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb64d9-46b7-46ec-9f5c-82d1c0e5e6b9 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:23.325466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:23.387574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:23.413614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eb64d9-46b7-46ec-9f5c-82d1c0e5e6b9 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:23.457750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a392345-9e0c-4778-b589-9f61a06c2493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:23.458958 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a392345-9e0c-4778-b589-9f61a06c2493 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:23.459235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a392345-9e0c-4778-b589-9f61a06c2493 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:23.459503 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a392345-9e0c-4778-b589-9f61a06c2493 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:23.472452 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278bbfd-e0f3-4d59-a4e9-f291a66c9e58 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:23.475613 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a392345-9e0c-4778-b589-9f61a06c2493 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:23.481688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-8a392345-9e0c-4778-b589-9f61a06c2493 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:23.481688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:23.481688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:23.481688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:23.481688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:23.481688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:23.481688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:23.481688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:23.481688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:23.481688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:23.481688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:23.481688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:23.481688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:23.481688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:23.481688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:23.481688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:23.482616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:23.482616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:23.482616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:23.482616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:23.482616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:23.482616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:23.482616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:23.482616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:23.482616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:23.482616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:23.482616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:23.482616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:23.482616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:23.482616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:23.482616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:23.482616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:23.483413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:23.483413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:23.483413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:23.483413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:23.483413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:23.483413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:23.483413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:23.483413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:23.483413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:23.483413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:23.483413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:23.483413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:23.484042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 187/362] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:23.489581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a392345-9e0c-4778-b589-9f61a06c2493 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:23.514576 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d9d3e-682a-444b-acbc-fc460974e697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:23.515438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d9d3e-682a-444b-acbc-fc460974e697 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:23.515718 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d9d3e-682a-444b-acbc-fc460974e697 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:23.515987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d9d3e-682a-444b-acbc-fc460974e697 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:23.516727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d9d3e-682a-444b-acbc-fc460974e697 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:23.517025 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d9d3e-682a-444b-acbc-fc460974e697 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:23.518387 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d9d3e-682a-444b-acbc-fc460974e697 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:23.529802 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 176/363] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:23.536977 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d9d3e-682a-444b-acbc-fc460974e697 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:23.653616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d9d3e-682a-444b-acbc-fc460974e697 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:23.657801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a392345-9e0c-4778-b589-9f61a06c2493 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:23.666440 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:23.689048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:23.692505 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d4cda2-b58d-43fd-a634-c7b0c7e294a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:23.693163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4cda2-b58d-43fd-a634-c7b0c7e294a8 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:23.693448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4cda2-b58d-43fd-a634-c7b0c7e294a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:23.693722 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d4cda2-b58d-43fd-a634-c7b0c7e294a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:23.707786 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355997fc-1a30-436a-bd18-f9492a118a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:23.708918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355997fc-1a30-436a-bd18-f9492a118a64 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:23.709488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355997fc-1a30-436a-bd18-f9492a118a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:23.710064 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355997fc-1a30-436a-bd18-f9492a118a64 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:23.711158 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355997fc-1a30-436a-bd18-f9492a118a64 None admin] RBAC: Adding query filter params (name=member) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:23.711663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355997fc-1a30-436a-bd18-f9492a118a64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:23.713577 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355997fc-1a30-436a-bd18-f9492a118a64 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:23.724662 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 188/364] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:23.726587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a94cf-3a5a-44de-9f19-2a568ddcdf37 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:23.727123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a94cf-3a5a-44de-9f19-2a568ddcdf37 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:23.727591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a94cf-3a5a-44de-9f19-2a568ddcdf37 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:23.729502 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 189/365] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:23.730513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a94cf-3a5a-44de-9f19-2a568ddcdf37 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:23.773284 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4cda2-b58d-43fd-a634-c7b0c7e294a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:23.774450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d4cda2-b58d-43fd-a634-c7b0c7e294a8 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:23.848053 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a94cf-3a5a-44de-9f19-2a568ddcdf37 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:23.887609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9dda71-b681-4b50-9c22-dba73e20b0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:23.888386 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 177/366] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:23.889239 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dda71-b681-4b50-9c22-dba73e20b0f4 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:23.889718 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dda71-b681-4b50-9c22-dba73e20b0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:23.889820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b7b5f-5ca4-48b0-81a0-9dfebaeae731 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:23.890104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b7b5f-5ca4-48b0-81a0-9dfebaeae731 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:23.890376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b7b5f-5ca4-48b0-81a0-9dfebaeae731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:23.890697 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dda71-b681-4b50-9c22-dba73e20b0f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:23.919292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-066b7b5f-5ca4-48b0-81a0-9dfebaeae731 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:23.920437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-066b7b5f-5ca4-48b0-81a0-9dfebaeae731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:23.922949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:23.925467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:23.993033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9dda71-b681-4b50-9c22-dba73e20b0f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:23.994907 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9dda71-b681-4b50-9c22-dba73e20b0f4 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:24.015191 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 178/367] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:24.016340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b7b5f-5ca4-48b0-81a0-9dfebaeae731 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:24.113360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 190/368] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:24.114564 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9dda71-b681-4b50-9c22-dba73e20b0f4 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:24.157677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b7b5f-5ca4-48b0-81a0-9dfebaeae731 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:24.178285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:24.204906 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9357d-dc35-4f03-a300-30c1c2dafe70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:24.205951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9357d-dc35-4f03-a300-30c1c2dafe70 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:24.207009 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9357d-dc35-4f03-a300-30c1c2dafe70 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:24.207523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9357d-dc35-4f03-a300-30c1c2dafe70 None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:24.209154 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9357d-dc35-4f03-a300-30c1c2dafe70 None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:24.210941 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9357d-dc35-4f03-a300-30c1c2dafe70 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:24.213288 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 179/369] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:24 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:24.214409 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9357d-dc35-4f03-a300-30c1c2dafe70 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:24.231527 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:24.245306 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9dda71-b681-4b50-9c22-dba73e20b0f4 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:24.293262 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1627087-d00d-46ac-a2ba-65b5ad7792a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:24.293878 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1627087-d00d-46ac-a2ba-65b5ad7792a6 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:24.294152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1627087-d00d-46ac-a2ba-65b5ad7792a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:24.294592 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1627087-d00d-46ac-a2ba-65b5ad7792a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:24.310772 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1627087-d00d-46ac-a2ba-65b5ad7792a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:24.318351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-d1627087-d00d-46ac-a2ba-65b5ad7792a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:04:24.318351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:24.318351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:24.318351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:24.318351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:24.318351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:24.318351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:24.318351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:24.318351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:24.318351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:24.318351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:24.318351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:24.318351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:24.318351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:24.318351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:24.318351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:24.319323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:24.319323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:24.319323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:24.319323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:24.319323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:24.319323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:24.319323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:24.319323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:24.319323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:24.319323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:24.319323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:24.319323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:24.319323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:24.319323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:24.319323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:24.319323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:24.320109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:24.320109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:24.320109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:24.320109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:24.320109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:24.320109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:24.320109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:24.320109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:24.320109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:24.320109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:24.320109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:04:24.320109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:24.320956 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 191/370] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:04:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:24.321759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1627087-d00d-46ac-a2ba-65b5ad7792a6 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:24.370197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9357d-dc35-4f03-a300-30c1c2dafe70 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:24.411675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c028e2a-6e18-4e04-95be-3740411bd10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:24.412322 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c028e2a-6e18-4e04-95be-3740411bd10a None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:24.412603 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c028e2a-6e18-4e04-95be-3740411bd10a None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:24.412925 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c028e2a-6e18-4e04-95be-3740411bd10a None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:24.414144 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c028e2a-6e18-4e04-95be-3740411bd10a None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:24.415604 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c028e2a-6e18-4e04-95be-3740411bd10a None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:24.417532 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 180/371] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:24 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:24.418135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c028e2a-6e18-4e04-95be-3740411bd10a None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:24.451675 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1627087-d00d-46ac-a2ba-65b5ad7792a6 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:24.476976 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:24.497001 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ea611-5b46-42bd-bcee-1c3325598733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:24.497918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ea611-5b46-42bd-bcee-1c3325598733 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:24.498209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ea611-5b46-42bd-bcee-1c3325598733 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:24.498487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ea611-5b46-42bd-bcee-1c3325598733 None admin] PATH_INFO: `/v3/groups/a51f0767088e4c889d3c33d8675952ec` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:24.499688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ea611-5b46-42bd-bcee-1c3325598733 None admin] RBAC: Authorizing `identity:get_group(group_id=a51f0767088e4c889d3c33d8675952ec)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:24.501193 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ea611-5b46-42bd-bcee-1c3325598733 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:24.504437 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 192/372] 10.222.0.45 () {56 vars in 1180 bytes} [Thu Aug 4 02:04:24 2022] GET /identity/v3/groups/a51f0767088e4c889d3c33d8675952ec => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:24.505338 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ea611-5b46-42bd-bcee-1c3325598733 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:24.521373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:24.556964 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c028e2a-6e18-4e04-95be-3740411bd10a None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:24.609405 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8b7f5-e073-4b86-bb51-77a2f4e4818c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:24.610121 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8b7f5-e073-4b86-bb51-77a2f4e4818c None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:24.610434 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8b7f5-e073-4b86-bb51-77a2f4e4818c None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:24.610706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8b7f5-e073-4b86-bb51-77a2f4e4818c None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:24.611570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df8b7f5-e073-4b86-bb51-77a2f4e4818c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:24.611928 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df8b7f5-e073-4b86-bb51-77a2f4e4818c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:24.614920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df8b7f5-e073-4b86-bb51-77a2f4e4818c None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:24.632679 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 181/373] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:24.637683 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8b7f5-e073-4b86-bb51-77a2f4e4818c None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:24.657782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ea611-5b46-42bd-bcee-1c3325598733 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:24.698212 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f700e7-5626-4217-a533-2bb4b909ada7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:24.698668 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f700e7-5626-4217-a533-2bb4b909ada7 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:24.698949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f700e7-5626-4217-a533-2bb4b909ada7 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:24.699223 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f700e7-5626-4217-a533-2bb4b909ada7 None admin] PATH_INFO: `/v3/groups/a51f0767088e4c889d3c33d8675952ec` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:24.700422 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f700e7-5626-4217-a533-2bb4b909ada7 None admin] RBAC: Authorizing `identity:get_group(group_id=a51f0767088e4c889d3c33d8675952ec)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:24.702102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f700e7-5626-4217-a533-2bb4b909ada7 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:24.704010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 193/374] 10.222.0.45 () {56 vars in 1180 bytes} [Thu Aug 4 02:04:24 2022] GET /identity/v3/groups/a51f0767088e4c889d3c33d8675952ec => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:24.704894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f700e7-5626-4217-a533-2bb4b909ada7 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:24.778306 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:24.788863 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8b7f5-e073-4b86-bb51-77a2f4e4818c None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:24.839388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6d238-a45d-46f0-ad69-c9789a3007c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:24.840034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6d238-a45d-46f0-ad69-c9789a3007c1 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:24.840306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6d238-a45d-46f0-ad69-c9789a3007c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:24.840579 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6d238-a45d-46f0-ad69-c9789a3007c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:24.841397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6d238-a45d-46f0-ad69-c9789a3007c1 None admin] RBAC: Adding query filter params (group.id=a51f0767088e4c889d3c33d8675952ec, role.id=80e5528d7c8742f78e29e64c1216176c, scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:24.841702 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6d238-a45d-46f0-ad69-c9789a3007c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:24.843163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6d238-a45d-46f0-ad69-c9789a3007c1 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:24.851399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 182/375] 10.222.0.45 () {56 vars in 1376 bytes} [Thu Aug 4 02:04:24 2022] GET /identity/v3/role_assignments?group.id=a51f0767088e4c889d3c33d8675952ec&scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92&role.id=80e5528d7c8742f78e29e64c1216176c => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:24.852288 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6d238-a45d-46f0-ad69-c9789a3007c1 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:24.855363 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:24.860239 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f700e7-5626-4217-a533-2bb4b909ada7 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:24.980961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fdd34-aad2-4c93-a1fd-be4dc4c146b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:24.982169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fdd34-aad2-4c93-a1fd-be4dc4c146b9 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:24.982674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fdd34-aad2-4c93-a1fd-be4dc4c146b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:24.983155 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fdd34-aad2-4c93-a1fd-be4dc4c146b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:25.019601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6d238-a45d-46f0-ad69-c9789a3007c1 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:25.090496 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8392dbe-b600-45c7-82c3-e4e5d7fdfc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:25.091520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8392dbe-b600-45c7-82c3-e4e5d7fdfc5f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:25.092006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8392dbe-b600-45c7-82c3-e4e5d7fdfc5f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:25.092482 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8392dbe-b600-45c7-82c3-e4e5d7fdfc5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:25.093524 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8392dbe-b600-45c7-82c3-e4e5d7fdfc5f None admin] RBAC: Adding query filter params (group.id=a51f0767088e4c889d3c33d8675952ec, role.id=8fc75c351a0349e294deeed1ff391c4c, scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:25.094039 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8392dbe-b600-45c7-82c3-e4e5d7fdfc5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:25.097976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8392dbe-b600-45c7-82c3-e4e5d7fdfc5f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:25.105739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 183/376] 10.222.0.45 () {56 vars in 1376 bytes} [Thu Aug 4 02:04:24 2022] GET /identity/v3/role_assignments?group.id=a51f0767088e4c889d3c33d8675952ec&scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92&role.id=8fc75c351a0349e294deeed1ff391c4c => generated 260 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:25.118865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1fdd34-aad2-4c93-a1fd-be4dc4c146b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:25.120367 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1fdd34-aad2-4c93-a1fd-be4dc4c146b9 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:25.196368 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:25.257627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 194/377] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:25.263532 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8392dbe-b600-45c7-82c3-e4e5d7fdfc5f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:25.347237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:25.389945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8392dbe-b600-45c7-82c3-e4e5d7fdfc5f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:25.431846 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9d891b-969c-4e3e-a428-4f599be0cdca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:25.432500 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d891b-969c-4e3e-a428-4f599be0cdca None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:25.432806 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d891b-969c-4e3e-a428-4f599be0cdca None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:25.433081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9d891b-969c-4e3e-a428-4f599be0cdca None admin] PATH_INFO: `/v3/projects/457cd62a685649778d5130f3937b1bcf` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:25.437379 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d891b-969c-4e3e-a428-4f599be0cdca None admin] RBAC: Authorizing `identity:get_project(project_id=457cd62a685649778d5130f3937b1bcf)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:25.438852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9d891b-969c-4e3e-a428-4f599be0cdca None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:25.440857 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 184/378] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:25 2022] GET /identity/v3/projects/457cd62a685649778d5130f3937b1bcf => generated 344 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:25.446421 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fdd34-aad2-4c93-a1fd-be4dc4c146b9 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:25.473351 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d816c93-aa6b-4d30-a3cb-96142cd410bd None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:25.473658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d816c93-aa6b-4d30-a3cb-96142cd410bd None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:25.473902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d816c93-aa6b-4d30-a3cb-96142cd410bd None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:25.475534 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 185/379] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:25.492899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfb2f5-6a53-4e0f-9294-16712c73cbc4 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:25.493208 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfb2f5-6a53-4e0f-9294-16712c73cbc4 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:25.493453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfb2f5-6a53-4e0f-9294-16712c73cbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:25.549939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-e6dfb2f5-6a53-4e0f-9294-16712c73cbc4 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:25.551957 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e6dfb2f5-6a53-4e0f-9294-16712c73cbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:25.574231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fdd34-aad2-4c93-a1fd-be4dc4c146b9 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:25.613258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91fa2c-3e35-4b53-88a0-245ec2b4ac3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:25.613903 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91fa2c-3e35-4b53-88a0-245ec2b4ac3c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:25.614176 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91fa2c-3e35-4b53-88a0-245ec2b4ac3c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:25.614440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f91fa2c-3e35-4b53-88a0-245ec2b4ac3c None admin] PATH_INFO: `/v3/groups/223a249772584fa892f68c98d61a85e3` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:25.621126 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91fa2c-3e35-4b53-88a0-245ec2b4ac3c None admin] RBAC: Authorizing `identity:get_group(group_id=223a249772584fa892f68c98d61a85e3)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:25.622592 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f91fa2c-3e35-4b53-88a0-245ec2b4ac3c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:25.624595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 195/380] 10.222.0.45 () {56 vars in 1180 bytes} [Thu Aug 4 02:04:25 2022] GET /identity/v3/groups/223a249772584fa892f68c98d61a85e3 => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:25.630709 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91fa2c-3e35-4b53-88a0-245ec2b4ac3c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:25.677045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:25.745973 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f91fa2c-3e35-4b53-88a0-245ec2b4ac3c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:25.801418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f789142-fe87-4e7b-96da-58726d8d9dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:25.801969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f789142-fe87-4e7b-96da-58726d8d9dac None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:25.802245 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f789142-fe87-4e7b-96da-58726d8d9dac None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:25.802511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f789142-fe87-4e7b-96da-58726d8d9dac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:25.803312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f789142-fe87-4e7b-96da-58726d8d9dac None admin] RBAC: Adding query filter params (group.id=223a249772584fa892f68c98d61a85e3, role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=457cd62a685649778d5130f3937b1bcf) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:25.803618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f789142-fe87-4e7b-96da-58726d8d9dac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:25.805182 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 186/381] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 04 02:04:25.805962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f789142-fe87-4e7b-96da-58726d8d9dac None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:25.816776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 196/382] 10.222.0.45 () {56 vars in 1376 bytes} [Thu Aug 4 02:04:25 2022] GET /identity/v3/role_assignments?group.id=223a249772584fa892f68c98d61a85e3&scope.project.id=457cd62a685649778d5130f3937b1bcf&role.id=35205394c8af4c52a61b12f92ba16644 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:25.849549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8729436-d46a-45f7-9661-5058e8076187 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:25.849861 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8729436-d46a-45f7-9661-5058e8076187 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:25.850092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8729436-d46a-45f7-9661-5058e8076187 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:25.851848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 187/383] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:04:25.859190 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f789142-fe87-4e7b-96da-58726d8d9dac None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:25.863877 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c93f0e-de21-457f-97d1-6411a783fe92 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:25.864181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c93f0e-de21-457f-97d1-6411a783fe92 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:25.864418 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c93f0e-de21-457f-97d1-6411a783fe92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:25.897184 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-00c93f0e-de21-457f-97d1-6411a783fe92 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:25.898570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-00c93f0e-de21-457f-97d1-6411a783fe92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:26.035252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:26.045644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:26.051742 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 188/384] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:26.075047 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f789142-fe87-4e7b-96da-58726d8d9dac None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:26.082270 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c93f0e-de21-457f-97d1-6411a783fe92 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:26.142482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cc709-501b-4efe-ab2a-c13732d285a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:26.143527 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cc709-501b-4efe-ab2a-c13732d285a8 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:26.144583 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cc709-501b-4efe-ab2a-c13732d285a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:26.145082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cc709-501b-4efe-ab2a-c13732d285a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:26.177611 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cc709-501b-4efe-ab2a-c13732d285a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:26.183157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-d70cc709-501b-4efe-ab2a-c13732d285a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:26.183157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:26.183157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:26.183157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:26.183157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:26.183157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:26.183157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:26.183157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:26.183157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:26.183157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:26.183157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:26.183157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:26.183157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:26.183157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:26.183157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:26.183157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:26.184087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:26.184087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:26.184087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:26.184087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:26.184087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:26.184087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:26.184087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:26.184087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:26.184087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:26.184087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:26.184087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:26.184087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:26.184087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:26.184087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:26.184087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:26.184087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:26.184910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:26.184910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:26.184910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:26.184910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:26.184910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:26.184910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:26.184910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:26.184910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:26.184910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:26.184910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:26.184910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:26.184910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:26.188725 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 197/385] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:26.191605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cc709-501b-4efe-ab2a-c13732d285a8 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:26.242217 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c93f0e-de21-457f-97d1-6411a783fe92 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:26.302211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa49845-ee4f-481d-86b1-47270b78e7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:26.304120 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa49845-ee4f-481d-86b1-47270b78e7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:26.305325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa49845-ee4f-481d-86b1-47270b78e7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:26.305989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa49845-ee4f-481d-86b1-47270b78e7e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:26.336364 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa49845-ee4f-481d-86b1-47270b78e7e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:26.341406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-baa49845-ee4f-481d-86b1-47270b78e7e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:26.341406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:26.341406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:26.341406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:26.341406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:26.341406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:26.341406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:26.341406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:26.341406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:26.341406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:26.341406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:26.341406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:26.341406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:26.341406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:26.341406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:26.341406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:26.342283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:26.342283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:26.342283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:26.342283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:26.342283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:26.342283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:26.342283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:26.342283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:26.342283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:26.342283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:26.342283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:26.342283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:26.342283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:26.342283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:26.342283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:26.342283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:26.343104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:26.343104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:26.343104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:26.343104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:26.343104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:26.343104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:26.343104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:26.343104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:26.343104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:26.343104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:26.343104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:26.343104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:26.343104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 189/386] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:04:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:04:26.351386 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa49845-ee4f-481d-86b1-47270b78e7e0 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:26.413541 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cc709-501b-4efe-ab2a-c13732d285a8 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:26.463836 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c1e81-b273-48a5-9fae-55ed33eb7019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:26.464869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c1e81-b273-48a5-9fae-55ed33eb7019 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:26.465375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c1e81-b273-48a5-9fae-55ed33eb7019 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:26.465853 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c1e81-b273-48a5-9fae-55ed33eb7019 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:26.466884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c1e81-b273-48a5-9fae-55ed33eb7019 None admin] RBAC: Adding query filter params (name=member) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:26.467410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c1e81-b273-48a5-9fae-55ed33eb7019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:26.469365 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:26.470399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c1e81-b273-48a5-9fae-55ed33eb7019 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:26.478267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:26.482119 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 198/387] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:26.489898 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c1e81-b273-48a5-9fae-55ed33eb7019 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:26.553338 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa49845-ee4f-481d-86b1-47270b78e7e0 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:26.633343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c1e81-b273-48a5-9fae-55ed33eb7019 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:26.667185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28994164-5408-4797-beef-b28ddcc3b1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:26.667828 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28994164-5408-4797-beef-b28ddcc3b1a4 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:26.668105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28994164-5408-4797-beef-b28ddcc3b1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:26.668377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28994164-5408-4797-beef-b28ddcc3b1a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:26.669167 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28994164-5408-4797-beef-b28ddcc3b1a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:26.669472 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28994164-5408-4797-beef-b28ddcc3b1a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:26.670983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28994164-5408-4797-beef-b28ddcc3b1a4 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:26.679389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d68705-b0d2-4ab4-95ad-ed8d1f9629c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:26.680042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d68705-b0d2-4ab4-95ad-ed8d1f9629c8 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:26.680323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d68705-b0d2-4ab4-95ad-ed8d1f9629c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:26.680598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d68705-b0d2-4ab4-95ad-ed8d1f9629c8 None admin] PATH_INFO: `/v3/groups/a51f0767088e4c889d3c33d8675952ec` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:26.682657 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d68705-b0d2-4ab4-95ad-ed8d1f9629c8 None admin] RBAC: Authorizing `identity:get_group(group_id=a51f0767088e4c889d3c33d8675952ec)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:26.684095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d68705-b0d2-4ab4-95ad-ed8d1f9629c8 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:26.685723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 199/388] 10.222.0.45 () {56 vars in 1180 bytes} [Thu Aug 4 02:04:26 2022] GET /identity/v3/groups/a51f0767088e4c889d3c33d8675952ec => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:26.688166 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 190/389] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:26.692870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d68705-b0d2-4ab4-95ad-ed8d1f9629c8 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:26.694149 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28994164-5408-4797-beef-b28ddcc3b1a4 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:26.825076 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28994164-5408-4797-beef-b28ddcc3b1a4 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:26.845369 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:26.871375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:26.871774 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b14ddd-28d3-4ba3-88d3-976bd3d699ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:26.872874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b14ddd-28d3-4ba3-88d3-976bd3d699ba None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:26.873155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b14ddd-28d3-4ba3-88d3-976bd3d699ba None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:26.873476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b14ddd-28d3-4ba3-88d3-976bd3d699ba None admin] PATH_INFO: `/v3/groups/a51f0767088e4c889d3c33d8675952ec` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:26.875254 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b14ddd-28d3-4ba3-88d3-976bd3d699ba None admin] RBAC: Authorizing `identity:get_group(group_id=a51f0767088e4c889d3c33d8675952ec)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:26.876955 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b14ddd-28d3-4ba3-88d3-976bd3d699ba None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:26.879429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 191/390] 10.222.0.45 () {56 vars in 1180 bytes} [Thu Aug 4 02:04:26 2022] GET /identity/v3/groups/a51f0767088e4c889d3c33d8675952ec => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:26.885835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b14ddd-28d3-4ba3-88d3-976bd3d699ba None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:26.970797 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d68705-b0d2-4ab4-95ad-ed8d1f9629c8 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:27.002837 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b14ddd-28d3-4ba3-88d3-976bd3d699ba None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:27.044114 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc54bd8-7c51-48b0-b9cc-91d8e1715c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:27.045285 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc54bd8-7c51-48b0-b9cc-91d8e1715c0c None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:27.045582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc54bd8-7c51-48b0-b9cc-91d8e1715c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:27.045863 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc54bd8-7c51-48b0-b9cc-91d8e1715c0c None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:27.047812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc54bd8-7c51-48b0-b9cc-91d8e1715c0c None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:27.051067 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc54bd8-7c51-48b0-b9cc-91d8e1715c0c None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:27.053196 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 192/391] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:26 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:27.058896 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc54bd8-7c51-48b0-b9cc-91d8e1715c0c None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:27.060534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707e1e9-cabb-4186-81b1-a2484795fe9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:27.062057 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707e1e9-cabb-4186-81b1-a2484795fe9b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:27.062539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707e1e9-cabb-4186-81b1-a2484795fe9b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:27.063011 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707e1e9-cabb-4186-81b1-a2484795fe9b None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:27.069391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707e1e9-cabb-4186-81b1-a2484795fe9b None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:27.071120 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707e1e9-cabb-4186-81b1-a2484795fe9b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:27.073434 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 200/392] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:26 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:27.078764 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707e1e9-cabb-4186-81b1-a2484795fe9b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:27.120480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:27.184645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc54bd8-7c51-48b0-b9cc-91d8e1715c0c None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:27.221969 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0143b365-a60a-4ad2-b61a-cd9b89eaab83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:27.223103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143b365-a60a-4ad2-b61a-cd9b89eaab83 None admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:27.223393 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143b365-a60a-4ad2-b61a-cd9b89eaab83 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:27.223669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143b365-a60a-4ad2-b61a-cd9b89eaab83 None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef/groups/a51f0767088e4c889d3c33d8675952ec/roles/80e5528d7c8742f78e29e64c1216176c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:27.226197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0143b365-a60a-4ad2-b61a-cd9b89eaab83 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f4f29d6d3a14c9eb0650231a82bbdef, group_id=a51f0767088e4c889d3c33d8675952ec, role_id=80e5528d7c8742f78e29e64c1216176c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:27.227915 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0143b365-a60a-4ad2-b61a-cd9b89eaab83 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:27.245429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 193/393] 10.222.0.45 () {58 vars in 1442 bytes} [Thu Aug 4 02:04:27 2022] PUT /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef/groups/a51f0767088e4c889d3c33d8675952ec/roles/80e5528d7c8742f78e29e64c1216176c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:04:27.306890 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707e1e9-cabb-4186-81b1-a2484795fe9b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:27.361800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880cebe-01d5-4b1d-a577-a8c783491c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:27.362947 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880cebe-01d5-4b1d-a577-a8c783491c78 None admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:27.363226 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880cebe-01d5-4b1d-a577-a8c783491c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:27.363494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880cebe-01d5-4b1d-a577-a8c783491c78 None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef/groups/a51f0767088e4c889d3c33d8675952ec/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:27.365623 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880cebe-01d5-4b1d-a577-a8c783491c78 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f4f29d6d3a14c9eb0650231a82bbdef, group_id=a51f0767088e4c889d3c33d8675952ec, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:27.367090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880cebe-01d5-4b1d-a577-a8c783491c78 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:27.373778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:27.392572 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 201/394] 10.222.0.45 () {58 vars in 1442 bytes} [Thu Aug 4 02:04:27 2022] PUT /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef/groups/a51f0767088e4c889d3c33d8675952ec/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:04:27.570779 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb5942-8738-43bb-a37d-e659146ab952 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:27.571076 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb5942-8738-43bb-a37d-e659146ab952 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:27.571330 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fb5942-8738-43bb-a37d-e659146ab952 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:27.573139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 194/395] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:27.579222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7fe4d-d667-4bb8-92f7-8248fa6df996 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:27.579568 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7fe4d-d667-4bb8-92f7-8248fa6df996 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:27.579774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7fe4d-d667-4bb8-92f7-8248fa6df996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:27.602461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-18d7fe4d-d667-4bb8-92f7-8248fa6df996 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:27.603679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-18d7fe4d-d667-4bb8-92f7-8248fa6df996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:27.687016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 202/396] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:27.714176 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb5942-8738-43bb-a37d-e659146ab952 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:27.738488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04f3c0-cc28-43ca-be26-8f3b9dc1f240 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:27.738810 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04f3c0-cc28-43ca-be26-8f3b9dc1f240 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:27.739023 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04f3c0-cc28-43ca-be26-8f3b9dc1f240 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:27.740519 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 203/397] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:27.752225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0cdd3-45e5-4074-bf40-91096cf69533 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:27.752531 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0cdd3-45e5-4074-bf40-91096cf69533 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:27.752840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a0cdd3-45e5-4074-bf40-91096cf69533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:27.778980 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c2a0cdd3-45e5-4074-bf40-91096cf69533 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:27.780178 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c2a0cdd3-45e5-4074-bf40-91096cf69533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:27.870223 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 204/398] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:27.901607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0cdd3-45e5-4074-bf40-91096cf69533 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:27.906388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fb5942-8738-43bb-a37d-e659146ab952 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:27.938956 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:27.978127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:27.986553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774a583-a07c-416e-af13-5923f9128cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:27.987214 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774a583-a07c-416e-af13-5923f9128cdc None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:27.990212 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774a583-a07c-416e-af13-5923f9128cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:27.990616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d774a583-a07c-416e-af13-5923f9128cdc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:28.014869 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d774a583-a07c-416e-af13-5923f9128cdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:28.025194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-d774a583-a07c-416e-af13-5923f9128cdc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:28.025194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:28.025194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:28.025194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:28.025194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:28.025194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:28.025194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:28.025194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:28.025194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:28.025194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:28.025194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:28.025194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:28.025194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:28.025194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:28.025194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:28.025194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:28.026247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:28.026247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:28.026247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:28.026247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:28.026247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:28.026247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:28.026247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:28.026247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:28.026247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:28.026247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:28.026247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:28.026247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:28.026247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:28.026247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:28.026247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:28.026247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:28.027080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:28.027080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:28.027080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:28.027080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:28.027080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:28.027080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:28.027080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:28.027080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:28.027080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:28.027080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:28.027080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:28.027080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:28.028875 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 195/399] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:04:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:28.041603 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774a583-a07c-416e-af13-5923f9128cdc None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:28.073769 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a0cdd3-45e5-4074-bf40-91096cf69533 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:28.136530 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e330f2-5eeb-4772-a6d5-9916cadf1881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:28.137277 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e330f2-5eeb-4772-a6d5-9916cadf1881 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:28.137558 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e330f2-5eeb-4772-a6d5-9916cadf1881 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:28.137832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e330f2-5eeb-4772-a6d5-9916cadf1881 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:28.150917 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e330f2-5eeb-4772-a6d5-9916cadf1881 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:28.157684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-58e330f2-5eeb-4772-a6d5-9916cadf1881 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:28.157684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:28.157684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:28.157684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:28.157684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:28.157684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:28.157684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:28.157684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:28.157684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:28.157684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:28.157684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:28.157684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:28.157684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:28.157684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:28.157684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:28.157684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:28.158603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:28.158603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:28.158603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:28.158603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:28.158603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:28.158603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:28.158603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:28.158603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:28.158603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:28.158603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:28.158603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:28.158603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:28.158603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:28.158603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:28.158603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:28.158603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:28.159479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:28.159479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:28.159479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:28.159479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:28.159479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:28.159479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:28.159479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:28.159479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:28.159479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:28.159479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:28.159479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:28.159479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:28.159479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 205/400] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:28.164946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e330f2-5eeb-4772-a6d5-9916cadf1881 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:28.271282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d774a583-a07c-416e-af13-5923f9128cdc None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:28.291082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e330f2-5eeb-4772-a6d5-9916cadf1881 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:28.308357 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:28.335666 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcbbbad-74c4-43a3-9dbb-5217187b754d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:28.336750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbbbad-74c4-43a3-9dbb-5217187b754d None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:28.337605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbbbad-74c4-43a3-9dbb-5217187b754d None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:28.338037 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcbbbad-74c4-43a3-9dbb-5217187b754d None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:28.338919 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbbbad-74c4-43a3-9dbb-5217187b754d None admin] RBAC: Adding query filter params (name=member) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:28.339378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbbbad-74c4-43a3-9dbb-5217187b754d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:28.341096 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcbbbad-74c4-43a3-9dbb-5217187b754d None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:28.356681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 206/401] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:28.359175 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcbbbad-74c4-43a3-9dbb-5217187b754d None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:28.377543 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849aaa2b-2b78-48de-9371-e41d66c64182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:28.378534 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849aaa2b-2b78-48de-9371-e41d66c64182 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:28.378817 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849aaa2b-2b78-48de-9371-e41d66c64182 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:28.379087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849aaa2b-2b78-48de-9371-e41d66c64182 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:28.379816 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849aaa2b-2b78-48de-9371-e41d66c64182 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:28.380140 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849aaa2b-2b78-48de-9371-e41d66c64182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:28.381532 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849aaa2b-2b78-48de-9371-e41d66c64182 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:28.396776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 196/402] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:28.402045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849aaa2b-2b78-48de-9371-e41d66c64182 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:28.489280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcbbbad-74c4-43a3-9dbb-5217187b754d None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:28.507251 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:28.534628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c5865b-872e-4dc1-a76c-402c69ff4c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:28.535712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5865b-872e-4dc1-a76c-402c69ff4c32 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:28.536484 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5865b-872e-4dc1-a76c-402c69ff4c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:28.537182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c5865b-872e-4dc1-a76c-402c69ff4c32 None admin] PATH_INFO: `/v3/groups/a51f0767088e4c889d3c33d8675952ec` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:28.538638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5865b-872e-4dc1-a76c-402c69ff4c32 None admin] RBAC: Authorizing `identity:get_group(group_id=a51f0767088e4c889d3c33d8675952ec)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:28.540776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c5865b-872e-4dc1-a76c-402c69ff4c32 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:28.543427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 207/403] 10.222.0.45 () {56 vars in 1180 bytes} [Thu Aug 4 02:04:28 2022] GET /identity/v3/groups/a51f0767088e4c889d3c33d8675952ec => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:28.544999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a38c9-2efe-44cc-aba2-7028f138cee9 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:28.545442 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a38c9-2efe-44cc-aba2-7028f138cee9 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:28.545870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a38c9-2efe-44cc-aba2-7028f138cee9 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:28.547246 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 208/404] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:28.556279 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e58f1-2b63-47fa-aeeb-4f0bab454016 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:28.556798 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e58f1-2b63-47fa-aeeb-4f0bab454016 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:28.557213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27e58f1-2b63-47fa-aeeb-4f0bab454016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:28.586237 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-d27e58f1-2b63-47fa-aeeb-4f0bab454016 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:28.587419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d27e58f1-2b63-47fa-aeeb-4f0bab454016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:28.600750 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849aaa2b-2b78-48de-9371-e41d66c64182 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:28.644678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:28.651972 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2a820-bfce-47eb-87eb-7ab8362a918e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:28.652319 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2a820-bfce-47eb-87eb-7ab8362a918e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:28.652591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2a820-bfce-47eb-87eb-7ab8362a918e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:28.656333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2a820-bfce-47eb-87eb-7ab8362a918e None admin] PATH_INFO: `/v3/groups/a51f0767088e4c889d3c33d8675952ec` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:28.657680 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f2a820-bfce-47eb-87eb-7ab8362a918e None admin] RBAC: Authorizing `identity:get_group(group_id=a51f0767088e4c889d3c33d8675952ec)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:28.659306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f2a820-bfce-47eb-87eb-7ab8362a918e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:28.661273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 197/405] 10.222.0.45 () {56 vars in 1180 bytes} [Thu Aug 4 02:04:28 2022] GET /identity/v3/groups/a51f0767088e4c889d3c33d8675952ec => generated 226 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:28.662057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2a820-bfce-47eb-87eb-7ab8362a918e None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:28.683981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 209/406] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:04:28.691555 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e58f1-2b63-47fa-aeeb-4f0bab454016 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:28.803174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2a820-bfce-47eb-87eb-7ab8362a918e None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:28.804802 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:28.828037 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27e58f1-2b63-47fa-aeeb-4f0bab454016 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:28.866360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11eab5a-cb62-4a53-bb57-9f6726588126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:28.867031 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11eab5a-cb62-4a53-bb57-9f6726588126 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:28.867313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11eab5a-cb62-4a53-bb57-9f6726588126 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:28.867588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11eab5a-cb62-4a53-bb57-9f6726588126 None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:28.871528 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11eab5a-cb62-4a53-bb57-9f6726588126 None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:28.873066 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11eab5a-cb62-4a53-bb57-9f6726588126 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:28.874876 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 198/407] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:28 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:28.875530 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbc526-2d21-4a78-8b3b-efe92e211c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:28.875668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11eab5a-cb62-4a53-bb57-9f6726588126 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:28.876393 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbc526-2d21-4a78-8b3b-efe92e211c2f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:28.876775 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbc526-2d21-4a78-8b3b-efe92e211c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:28.877111 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbc526-2d21-4a78-8b3b-efe92e211c2f None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:28.878452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbc526-2d21-4a78-8b3b-efe92e211c2f None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:28.880003 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbc526-2d21-4a78-8b3b-efe92e211c2f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:28.883221 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 210/408] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:28 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:28.884082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbc526-2d21-4a78-8b3b-efe92e211c2f None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:28.942812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:29.010680 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11eab5a-cb62-4a53-bb57-9f6726588126 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:29.020307 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbc526-2d21-4a78-8b3b-efe92e211c2f None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:29.058322 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f41a74-1cfe-439b-a8a0-4aa596f7ec42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:29.059209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f41a74-1cfe-439b-a8a0-4aa596f7ec42 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:29.059646 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f41a74-1cfe-439b-a8a0-4aa596f7ec42 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:29.059763 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2952be0-7e0f-41e8-8cd9-964d97518ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:29.060175 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f41a74-1cfe-439b-a8a0-4aa596f7ec42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:29.060401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2952be0-7e0f-41e8-8cd9-964d97518ec3 None admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:29.060714 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2952be0-7e0f-41e8-8cd9-964d97518ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:29.061232 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2952be0-7e0f-41e8-8cd9-964d97518ec3 None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92/groups/a51f0767088e4c889d3c33d8675952ec/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:29.063158 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2952be0-7e0f-41e8-8cd9-964d97518ec3 None admin] RBAC: Authorizing `identity:create_grant(project_id=c872c66e5ba64576ab2eac8ce92c8b92, group_id=a51f0767088e4c889d3c33d8675952ec, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:29.064569 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2952be0-7e0f-41e8-8cd9-964d97518ec3 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:29.077852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f41a74-1cfe-439b-a8a0-4aa596f7ec42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:29.090311 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 211/409] 10.222.0.45 () {58 vars in 1442 bytes} [Thu Aug 4 02:04:28 2022] PUT /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92/groups/a51f0767088e4c889d3c33d8675952ec/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:04:29.090915 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2952be0-7e0f-41e8-8cd9-964d97518ec3 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:29.092757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-83f41a74-1cfe-439b-a8a0-4aa596f7ec42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:04:29.092757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:29.092757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:29.092757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:29.092757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:29.092757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:29.092757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:29.092757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:29.092757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:29.092757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:29.092757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:29.092757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:29.092757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:29.092757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:29.092757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:29.092757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:29.093684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:29.093684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:29.093684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:29.093684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:29.093684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:29.093684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:29.093684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:29.093684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:29.093684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:29.093684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:29.093684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:29.093684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:29.093684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:29.093684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:29.093684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:29.093684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:29.094488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:29.094488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:29.094488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:29.094488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:29.094488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:29.094488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:29.094488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:29.094488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:29.094488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:29.094488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:29.094488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:04:29.094488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:29.094488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 199/410] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:04:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:29.102071 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f41a74-1cfe-439b-a8a0-4aa596f7ec42 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:29.160512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:29.277882 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2952be0-7e0f-41e8-8cd9-964d97518ec3 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:29.280951 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:29.325939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f9669-940d-429e-97f6-6c1c5262b45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:29.326985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f9669-940d-429e-97f6-6c1c5262b45b None admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:29.327485 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f9669-940d-429e-97f6-6c1c5262b45b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:29.327969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f9669-940d-429e-97f6-6c1c5262b45b None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92/groups/a51f0767088e4c889d3c33d8675952ec/roles/80e5528d7c8742f78e29e64c1216176c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:29.330933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f9669-940d-429e-97f6-6c1c5262b45b None admin] RBAC: Authorizing `identity:create_grant(project_id=c872c66e5ba64576ab2eac8ce92c8b92, group_id=a51f0767088e4c889d3c33d8675952ec, role_id=80e5528d7c8742f78e29e64c1216176c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:29.332901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947f9669-940d-429e-97f6-6c1c5262b45b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:29.349986 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 212/411] 10.222.0.45 () {58 vars in 1442 bytes} [Thu Aug 4 02:04:29 2022] PUT /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92/groups/a51f0767088e4c889d3c33d8675952ec/roles/80e5528d7c8742f78e29e64c1216176c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:04:29.354674 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f41a74-1cfe-439b-a8a0-4aa596f7ec42 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:29.403406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cbf380-b3d5-48f8-988b-dccac70b9813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:29.404457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbf380-b3d5-48f8-988b-dccac70b9813 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:29.404975 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbf380-b3d5-48f8-988b-dccac70b9813 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:29.405461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cbf380-b3d5-48f8-988b-dccac70b9813 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:29.406450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbf380-b3d5-48f8-988b-dccac70b9813 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:29.407007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbf380-b3d5-48f8-988b-dccac70b9813 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:29.408789 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cbf380-b3d5-48f8-988b-dccac70b9813 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:29.420551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 200/412] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:29.426620 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f9669-940d-429e-97f6-6c1c5262b45b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:29.542656 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947f9669-940d-429e-97f6-6c1c5262b45b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:29.584638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d03d2d-2c37-4dd3-b417-96faed0f6b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:29.585214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d03d2d-2c37-4dd3-b417-96faed0f6b85 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:29.585489 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d03d2d-2c37-4dd3-b417-96faed0f6b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:29.585760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d03d2d-2c37-4dd3-b417-96faed0f6b85 None admin] PATH_INFO: `/v3/groups/223a249772584fa892f68c98d61a85e3` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:29.587529 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d03d2d-2c37-4dd3-b417-96faed0f6b85 None admin] RBAC: Authorizing `identity:get_group(group_id=223a249772584fa892f68c98d61a85e3)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:29.588986 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d03d2d-2c37-4dd3-b417-96faed0f6b85 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:29.590899 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 213/413] 10.222.0.45 () {56 vars in 1180 bytes} [Thu Aug 4 02:04:29 2022] GET /identity/v3/groups/223a249772584fa892f68c98d61a85e3 => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:29.596551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cbf380-b3d5-48f8-988b-dccac70b9813 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:29.723252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:29.734898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cbf380-b3d5-48f8-988b-dccac70b9813 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:29.773164 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca5395c-da41-4f73-bd10-faf53b60a0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:29.773825 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5395c-da41-4f73-bd10-faf53b60a0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:29.774104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5395c-da41-4f73-bd10-faf53b60a0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:29.774378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca5395c-da41-4f73-bd10-faf53b60a0e7 None admin] PATH_INFO: `/v3/projects/457cd62a685649778d5130f3937b1bcf` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:29.775554 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5395c-da41-4f73-bd10-faf53b60a0e7 None admin] RBAC: Authorizing `identity:get_project(project_id=457cd62a685649778d5130f3937b1bcf)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:29.778321 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca5395c-da41-4f73-bd10-faf53b60a0e7 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:29.780561 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 201/414] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:29 2022] GET /identity/v3/projects/457cd62a685649778d5130f3937b1bcf => generated 344 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:29.786811 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d03d2d-2c37-4dd3-b417-96faed0f6b85 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:29.836130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:29.911449 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676771f-ff5a-473f-86d0-45adfd58844c None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:29.912072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676771f-ff5a-473f-86d0-45adfd58844c None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:29.912638 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676771f-ff5a-473f-86d0-45adfd58844c None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:29.916167 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 202/415] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:29.921981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d03d2d-2c37-4dd3-b417-96faed0f6b85 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:29.923234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f018429-d9da-4a87-8f3c-0ca4ef26c404 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:29.923860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f018429-d9da-4a87-8f3c-0ca4ef26c404 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:29.924345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f018429-d9da-4a87-8f3c-0ca4ef26c404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:29.959305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ad8ae-a812-4e7a-8f87-c25bb807f986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:29.959979 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ad8ae-a812-4e7a-8f87-c25bb807f986 None admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:29.960256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ad8ae-a812-4e7a-8f87-c25bb807f986 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:29.960529 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ad8ae-a812-4e7a-8f87-c25bb807f986 None admin] PATH_INFO: `/v3/projects/457cd62a685649778d5130f3937b1bcf/groups/223a249772584fa892f68c98d61a85e3/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:29.961510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-0f018429-d9da-4a87-8f3c-0ca4ef26c404 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:29.962548 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4ad8ae-a812-4e7a-8f87-c25bb807f986 None admin] RBAC: Authorizing `identity:create_grant(project_id=457cd62a685649778d5130f3937b1bcf, group_id=223a249772584fa892f68c98d61a85e3, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:29.963418 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-0f018429-d9da-4a87-8f3c-0ca4ef26c404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:29.963986 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4ad8ae-a812-4e7a-8f87-c25bb807f986 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:29.984028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 214/416] 10.222.0.45 () {58 vars in 1442 bytes} [Thu Aug 4 02:04:29 2022] PUT /identity/v3/projects/457cd62a685649778d5130f3937b1bcf/groups/223a249772584fa892f68c98d61a85e3/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:04:30.028390 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45522722-6ff0-43a1-aa78-da4da878de89 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:30.028733 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45522722-6ff0-43a1-aa78-da4da878de89 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:30.028989 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45522722-6ff0-43a1-aa78-da4da878de89 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:30.031708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 215/417] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:04:30.042155 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e786a08-8b25-47b5-98c1-270ca02a0194 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:30.042462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e786a08-8b25-47b5-98c1-270ca02a0194 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:30.042717 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e786a08-8b25-47b5-98c1-270ca02a0194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:30.059265 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 203/418] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:30.089928 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-8e786a08-8b25-47b5-98c1-270ca02a0194 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:30.091084 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8e786a08-8b25-47b5-98c1-270ca02a0194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:30.109349 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f018429-d9da-4a87-8f3c-0ca4ef26c404 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:30.193063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 216/419] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:04:30.224054 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e786a08-8b25-47b5-98c1-270ca02a0194 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:30.246829 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:30.264558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:30.274556 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f018429-d9da-4a87-8f3c-0ca4ef26c404 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:30.317910 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b84d4-5522-4ec8-a97c-297885220b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:30.318710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b84d4-5522-4ec8-a97c-297885220b46 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:30.318995 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b84d4-5522-4ec8-a97c-297885220b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:30.319269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b84d4-5522-4ec8-a97c-297885220b46 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:30.331125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b84d4-5522-4ec8-a97c-297885220b46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:30.338951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-4a0b84d4-5522-4ec8-a97c-297885220b46 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:30.338951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:30.338951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:30.338951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:30.338951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:30.338951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:30.338951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:30.338951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:30.338951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:30.338951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:30.338951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:30.338951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:30.338951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:30.338951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:30.338951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:30.338951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:30.341110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:30.341110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:30.341110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:30.341110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:30.341110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:30.341110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:30.341110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:30.341110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:30.341110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:30.341110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:30.341110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:30.341110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:30.341110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:30.341110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:30.341110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:30.341110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:30.341897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:30.341897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:30.341897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:30.341897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:30.341897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:30.341897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:30.341897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:30.341897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:30.341897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:30.341897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:30.341897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:30.341897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:30.344745 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 204/420] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:04:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:04:30.346445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b84d4-5522-4ec8-a97c-297885220b46 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:30.412954 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e786a08-8b25-47b5-98c1-270ca02a0194 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:30.451320 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b84d4-5522-4ec8-a97c-297885220b46 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:30.474491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a65245-520a-4a60-a22a-4343cc296f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:30.475234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a65245-520a-4a60-a22a-4343cc296f7f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:30.475615 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a65245-520a-4a60-a22a-4343cc296f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:30.475895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a65245-520a-4a60-a22a-4343cc296f7f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:30.486118 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615798ef-c08e-4abc-b51c-4f4b49eae2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:30.486802 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615798ef-c08e-4abc-b51c-4f4b49eae2e4 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:30.487085 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615798ef-c08e-4abc-b51c-4f4b49eae2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:30.487918 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615798ef-c08e-4abc-b51c-4f4b49eae2e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:30.488681 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615798ef-c08e-4abc-b51c-4f4b49eae2e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:30.488995 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615798ef-c08e-4abc-b51c-4f4b49eae2e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:30.490461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615798ef-c08e-4abc-b51c-4f4b49eae2e4 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:30.504278 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 205/421] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:30.505599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615798ef-c08e-4abc-b51c-4f4b49eae2e4 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:30.505774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a65245-520a-4a60-a22a-4343cc296f7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:30.511016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-32a65245-520a-4a60-a22a-4343cc296f7f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:30.511016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:30.511016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:30.511016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:30.511016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:30.511016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:30.511016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:30.511016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:30.511016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:30.511016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:30.511016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:30.511016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:30.511016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:30.511016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:30.511016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:30.511016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:30.513089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:30.513089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:30.513089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:30.513089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:30.513089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:30.513089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:30.513089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:30.513089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:30.513089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:30.513089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:30.513089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:30.513089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:30.513089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:30.513089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:30.513089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:30.513089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:30.514974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:30.514974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:30.514974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:30.514974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:30.514974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:30.514974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:30.514974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:30.514974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:30.514974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:30.514974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:30.514974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:30.514974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:30.521169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 217/422] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:04:30.526586 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:30.528251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a65245-520a-4a60-a22a-4343cc296f7f None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:30.617705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615798ef-c08e-4abc-b51c-4f4b49eae2e4 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:30.661098 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c4b0f-228e-40cb-b87e-7c2c8d1ee12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:30.661727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c4b0f-228e-40cb-b87e-7c2c8d1ee12f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:30.662010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c4b0f-228e-40cb-b87e-7c2c8d1ee12f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:30.662285 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c4b0f-228e-40cb-b87e-7c2c8d1ee12f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:30.735264 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c4b0f-228e-40cb-b87e-7c2c8d1ee12f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:30.736834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c4b0f-228e-40cb-b87e-7c2c8d1ee12f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:30.761017 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:30.790937 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:30.845657 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a65245-520a-4a60-a22a-4343cc296f7f None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:30.867605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 206/423] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 02:04:30.869929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c4b0f-228e-40cb-b87e-7c2c8d1ee12f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:30.915719 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a95c8-c18c-4114-856e-90c631d22ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:30.916350 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a95c8-c18c-4114-856e-90c631d22ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:30.916642 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a95c8-c18c-4114-856e-90c631d22ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:30.916931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a95c8-c18c-4114-856e-90c631d22ae2 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:30.917654 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a95c8-c18c-4114-856e-90c631d22ae2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:30.917956 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a95c8-c18c-4114-856e-90c631d22ae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:30.919330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a95c8-c18c-4114-856e-90c631d22ae2 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:30.941342 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 218/424] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 414 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:30.947056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a95c8-c18c-4114-856e-90c631d22ae2 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:31.013106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c4b0f-228e-40cb-b87e-7c2c8d1ee12f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:31.028956 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:31.061476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de5744-978d-4d68-8035-a44125c8447c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:31.062130 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de5744-978d-4d68-8035-a44125c8447c None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:31.062411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de5744-978d-4d68-8035-a44125c8447c None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:31.062684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de5744-978d-4d68-8035-a44125c8447c None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:31.065439 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de5744-978d-4d68-8035-a44125c8447c None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:31.066916 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de5744-978d-4d68-8035-a44125c8447c None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:31.068639 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 207/425] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:30 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:31.074544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de5744-978d-4d68-8035-a44125c8447c None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:31.120771 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a95c8-c18c-4114-856e-90c631d22ae2 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:31.156025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed0ef2-a809-45cc-9c4c-cc09a57ae248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:31.156667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed0ef2-a809-45cc-9c4c-cc09a57ae248 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:31.156952 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed0ef2-a809-45cc-9c4c-cc09a57ae248 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:31.157224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed0ef2-a809-45cc-9c4c-cc09a57ae248 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:31.169637 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:31.183668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de5744-978d-4d68-8035-a44125c8447c None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:31.230823 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed0ef2-a809-45cc-9c4c-cc09a57ae248 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:31.232657 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed0ef2-a809-45cc-9c4c-cc09a57ae248 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:31.264069 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a63e22-8c5b-4baf-b70c-97d23162ac4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:31.264748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a63e22-8c5b-4baf-b70c-97d23162ac4d None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:31.265088 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a63e22-8c5b-4baf-b70c-97d23162ac4d None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:31.265309 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a63e22-8c5b-4baf-b70c-97d23162ac4d None admin] PATH_INFO: `/v3/groups/a51f0767088e4c889d3c33d8675952ec` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:31.269394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a63e22-8c5b-4baf-b70c-97d23162ac4d None admin] RBAC: Authorizing `identity:get_group(group_id=a51f0767088e4c889d3c33d8675952ec)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:31.270851 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a63e22-8c5b-4baf-b70c-97d23162ac4d None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:31.273463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 208/426] 10.222.0.45 () {56 vars in 1180 bytes} [Thu Aug 4 02:04:31 2022] GET /identity/v3/groups/a51f0767088e4c889d3c33d8675952ec => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:31.278370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a63e22-8c5b-4baf-b70c-97d23162ac4d None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:31.342614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 219/427] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 02:04:31.349593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed0ef2-a809-45cc-9c4c-cc09a57ae248 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:31.377101 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:31.438681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:31.483910 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a63e22-8c5b-4baf-b70c-97d23162ac4d None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:31.545288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed0ef2-a809-45cc-9c4c-cc09a57ae248 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:31.555828 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8b559-86ba-414c-a2ec-7b82ef463cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:31.556454 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8b559-86ba-414c-a2ec-7b82ef463cfc None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:31.556756 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8b559-86ba-414c-a2ec-7b82ef463cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:31.557028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8b559-86ba-414c-a2ec-7b82ef463cfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:31.557830 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8b559-86ba-414c-a2ec-7b82ef463cfc None admin] RBAC: Adding query filter params (group.id=a51f0767088e4c889d3c33d8675952ec, role.id=80e5528d7c8742f78e29e64c1216176c, scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:31.558136 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8b559-86ba-414c-a2ec-7b82ef463cfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:31.559575 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8b559-86ba-414c-a2ec-7b82ef463cfc None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:31.572349 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 209/428] 10.222.0.45 () {56 vars in 1376 bytes} [Thu Aug 4 02:04:31 2022] GET /identity/v3/role_assignments?group.id=a51f0767088e4c889d3c33d8675952ec&scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef&role.id=80e5528d7c8742f78e29e64c1216176c => generated 611 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:31.594002 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84799b-6162-4bd9-b9ec-031b7a6567b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:31.594850 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84799b-6162-4bd9-b9ec-031b7a6567b4 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:31.595228 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84799b-6162-4bd9-b9ec-031b7a6567b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:31.595599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd84799b-6162-4bd9-b9ec-031b7a6567b4 None admin] PATH_INFO: `/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:31.597599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd84799b-6162-4bd9-b9ec-031b7a6567b4 None admin] RBAC: Authorizing `identity:get_project(project_id=2f4f29d6d3a14c9eb0650231a82bbdef)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:31.599070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd84799b-6162-4bd9-b9ec-031b7a6567b4 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:31.601218 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 220/429] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:31 2022] GET /identity/v3/projects/2f4f29d6d3a14c9eb0650231a82bbdef => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:31.609639 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8b559-86ba-414c-a2ec-7b82ef463cfc None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:31.754592 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8b559-86ba-414c-a2ec-7b82ef463cfc None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:31.768349 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:31.793682 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc2d49e-7086-4329-937d-75c8b3693a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:31.794332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2d49e-7086-4329-937d-75c8b3693a56 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:31.794612 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2d49e-7086-4329-937d-75c8b3693a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:31.794888 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc2d49e-7086-4329-937d-75c8b3693a56 None admin] PATH_INFO: `/v3/groups/a51f0767088e4c889d3c33d8675952ec` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:31.796034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2d49e-7086-4329-937d-75c8b3693a56 None admin] RBAC: Authorizing `identity:get_group(group_id=a51f0767088e4c889d3c33d8675952ec)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:31.797570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc2d49e-7086-4329-937d-75c8b3693a56 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:31.799467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 210/430] 10.222.0.45 () {56 vars in 1180 bytes} [Thu Aug 4 02:04:31 2022] GET /identity/v3/groups/a51f0767088e4c889d3c33d8675952ec => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:31.805253 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84799b-6162-4bd9-b9ec-031b7a6567b4 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:31.810120 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f3e06-f4f0-4459-ba56-3350fc9caa34 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:31.810401 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f3e06-f4f0-4459-ba56-3350fc9caa34 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:31.810679 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f3e06-f4f0-4459-ba56-3350fc9caa34 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:31.812530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 211/431] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:31.817965 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eca618-49d6-43c3-a8a7-33c4c12fb1e3 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:31.818627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eca618-49d6-43c3-a8a7-33c4c12fb1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:31.819102 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eca618-49d6-43c3-a8a7-33c4c12fb1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:31.839443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-b9eca618-49d6-43c3-a8a7-33c4c12fb1e3 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:31.842933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-b9eca618-49d6-43c3-a8a7-33c4c12fb1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:31.930324 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd84799b-6162-4bd9-b9ec-031b7a6567b4 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:31.944161 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:31.954629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 212/432] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:31.978737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38271080-e18a-4f8d-95b5-6ed8bde80e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:31.979832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38271080-e18a-4f8d-95b5-6ed8bde80e2e None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:31.980312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38271080-e18a-4f8d-95b5-6ed8bde80e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:31.980510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eca618-49d6-43c3-a8a7-33c4c12fb1e3 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:31.980828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38271080-e18a-4f8d-95b5-6ed8bde80e2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:31.982314 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38271080-e18a-4f8d-95b5-6ed8bde80e2e None admin] RBAC: Adding query filter params (group.id=a51f0767088e4c889d3c33d8675952ec, role.id=8fc75c351a0349e294deeed1ff391c4c, scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:31.982842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38271080-e18a-4f8d-95b5-6ed8bde80e2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:31.986338 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38271080-e18a-4f8d-95b5-6ed8bde80e2e None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:31.996663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 221/433] 10.222.0.45 () {56 vars in 1376 bytes} [Thu Aug 4 02:04:31 2022] GET /identity/v3/role_assignments?group.id=a51f0767088e4c889d3c33d8675952ec&scope.project.id=2f4f29d6d3a14c9eb0650231a82bbdef&role.id=8fc75c351a0349e294deeed1ff391c4c => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:32.101512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eca618-49d6-43c3-a8a7-33c4c12fb1e3 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:32.113928 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:32.141454 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dde2e9-b653-427a-9a1b-02588b3918d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:32.142404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dde2e9-b653-427a-9a1b-02588b3918d5 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:32.142829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dde2e9-b653-427a-9a1b-02588b3918d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:32.143244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dde2e9-b653-427a-9a1b-02588b3918d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:32.156043 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dde2e9-b653-427a-9a1b-02588b3918d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:32.161382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-d4dde2e9-b653-427a-9a1b-02588b3918d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:32.161382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:32.161382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:32.161382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:32.161382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:32.161382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:32.161382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:32.161382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:32.161382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:32.161382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:32.161382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:32.161382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:32.161382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:32.161382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:32.161382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:32.161382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:32.162441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:32.162441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:32.162441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:32.162441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:32.162441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:32.162441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:32.162441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:32.162441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:32.162441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:32.162441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:32.162441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:32.162441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:32.162441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:32.162441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:32.162441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:32.162441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:32.163358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:32.163358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:32.163358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:32.163358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:32.163358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:32.163358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:32.163358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:32.163358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:32.163358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:32.163358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:32.163358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 04 02:04:32.163358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:32.165539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 213/434] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:32.172167 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38271080-e18a-4f8d-95b5-6ed8bde80e2e None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:32.285459 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38271080-e18a-4f8d-95b5-6ed8bde80e2e None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:32.309200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799f9b9-218e-4626-a42b-d707cd51f20b None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:32.309857 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799f9b9-218e-4626-a42b-d707cd51f20b None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:32.310343 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799f9b9-218e-4626-a42b-d707cd51f20b None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:32.312146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 214/435] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:32.319781 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868c202-78f8-44d5-baf6-d16d1e0587dc None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:32.320362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868c202-78f8-44d5-baf6-d16d1e0587dc None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:32.320670 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868c202-78f8-44d5-baf6-d16d1e0587dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:32.327327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302de67e-db1a-4f06-8eae-f1ef5164184f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:32.328075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302de67e-db1a-4f06-8eae-f1ef5164184f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:32.328358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302de67e-db1a-4f06-8eae-f1ef5164184f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:32.328940 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302de67e-db1a-4f06-8eae-f1ef5164184f None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:32.329923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302de67e-db1a-4f06-8eae-f1ef5164184f None admin] RBAC: Adding query filter params (name=member) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:32.330231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302de67e-db1a-4f06-8eae-f1ef5164184f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:32.332237 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302de67e-db1a-4f06-8eae-f1ef5164184f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:32.342498 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-2868c202-78f8-44d5-baf6-d16d1e0587dc None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:32.342860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 222/436] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:04:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:32.343553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2868c202-78f8-44d5-baf6-d16d1e0587dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:32.353944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302de67e-db1a-4f06-8eae-f1ef5164184f None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:32.398768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:32.443919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 215/437] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:32.467212 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868c202-78f8-44d5-baf6-d16d1e0587dc None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:32.484999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302de67e-db1a-4f06-8eae-f1ef5164184f None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:32.529594 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c89ef-699f-4335-acee-48abda789ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:32.530498 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c89ef-699f-4335-acee-48abda789ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:32.530779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c89ef-699f-4335-acee-48abda789ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:32.531052 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c89ef-699f-4335-acee-48abda789ea1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:32.595837 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:32.600686 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868c202-78f8-44d5-baf6-d16d1e0587dc None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:32.618419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c89ef-699f-4335-acee-48abda789ea1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:32.619874 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c89ef-699f-4335-acee-48abda789ea1 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:32.646594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75f3ea-784e-4ce8-bc2c-dc1b16ad3e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:32.646594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75f3ea-784e-4ce8-bc2c-dc1b16ad3e9d None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:32.646594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75f3ea-784e-4ce8-bc2c-dc1b16ad3e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:32.646594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75f3ea-784e-4ce8-bc2c-dc1b16ad3e9d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:32.659571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75f3ea-784e-4ce8-bc2c-dc1b16ad3e9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:32.660978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:32.666299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-fb75f3ea-784e-4ce8-bc2c-dc1b16ad3e9d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:32.666299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:32.666299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:32.666299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:32.666299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:32.666299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:32.666299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:32.666299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:32.666299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:32.666299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:32.666299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:32.666299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:32.666299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:32.666299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:32.666299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:32.666299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:32.667220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:32.667220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:32.667220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:32.667220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:32.667220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:32.667220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:32.667220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:32.667220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:32.667220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:32.667220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:32.667220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:32.667220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:32.667220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:32.667220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:32.667220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:32.667220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:32.668234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:32.668234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:32.668234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:32.668234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:32.668234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:32.668234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:32.668234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:32.668234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:32.668234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:32.668234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:32.668234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:32.668234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:32.668234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 216/438] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:04:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:32.669370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52162bdf-aaff-44c9-ad6b-aa52a5076405 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:32.669370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52162bdf-aaff-44c9-ad6b-aa52a5076405 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:32.669370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52162bdf-aaff-44c9-ad6b-aa52a5076405 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:32.670771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 217/439] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:32.674704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52162bdf-aaff-44c9-ad6b-aa52a5076405 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:32.735202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 223/440] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:32.736461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ff6c9-3652-4b7d-b1b8-d6ada26385cb None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:32.736776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ff6c9-3652-4b7d-b1b8-d6ada26385cb None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:32.737055 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ff6c9-3652-4b7d-b1b8-d6ada26385cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:32.755557 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-bf9ff6c9-3652-4b7d-b1b8-d6ada26385cb None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:32.756781 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-bf9ff6c9-3652-4b7d-b1b8-d6ada26385cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:32.779149 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52162bdf-aaff-44c9-ad6b-aa52a5076405 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:32.812727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76dae6-4ea0-4638-8be3-25370d442c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:32.813339 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76dae6-4ea0-4638-8be3-25370d442c82 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:32.813620 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76dae6-4ea0-4638-8be3-25370d442c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:32.813918 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76dae6-4ea0-4638-8be3-25370d442c82 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:32.814634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76dae6-4ea0-4638-8be3-25370d442c82 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:32.814939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76dae6-4ea0-4638-8be3-25370d442c82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:32.816882 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76dae6-4ea0-4638-8be3-25370d442c82 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:32.828020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 218/441] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:32.828020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76dae6-4ea0-4638-8be3-25370d442c82 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:32.835244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 224/442] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:32.836430 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ff6c9-3652-4b7d-b1b8-d6ada26385cb None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:32.852100 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:32.910087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:32.931577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76dae6-4ea0-4638-8be3-25370d442c82 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:32.944018 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ff6c9-3652-4b7d-b1b8-d6ada26385cb None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:32.965041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ad597-558a-498f-bde8-55b055d22d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:32.965677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ad597-558a-498f-bde8-55b055d22d5d None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:32.965954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ad597-558a-498f-bde8-55b055d22d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:32.966230 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ad597-558a-498f-bde8-55b055d22d5d None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:32.967389 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ad597-558a-498f-bde8-55b055d22d5d None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:32.968865 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ad597-558a-498f-bde8-55b055d22d5d None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:32.970510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 219/443] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:32 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:32.971313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ad597-558a-498f-bde8-55b055d22d5d None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:32.977951 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72eb72-7d41-4055-8944-16b16cc76b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:32.978577 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72eb72-7d41-4055-8944-16b16cc76b60 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:32.978858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72eb72-7d41-4055-8944-16b16cc76b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:32.979131 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a72eb72-7d41-4055-8944-16b16cc76b60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:33.051401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72eb72-7d41-4055-8944-16b16cc76b60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:33.052897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a72eb72-7d41-4055-8944-16b16cc76b60 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:33.066343 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:33.076446 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ad597-558a-498f-bde8-55b055d22d5d None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:33.109351 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8faec2-96d8-4ea1-baba-42f50c4a9b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:33.109911 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8faec2-96d8-4ea1-baba-42f50c4a9b9e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:33.110192 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8faec2-96d8-4ea1-baba-42f50c4a9b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:33.110469 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8faec2-96d8-4ea1-baba-42f50c4a9b9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:33.116255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:33.121120 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8faec2-96d8-4ea1-baba-42f50c4a9b9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:33.126417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-de8faec2-96d8-4ea1-baba-42f50c4a9b9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:04:33.126417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:33.126417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:33.126417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:33.126417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:33.126417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:33.126417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:33.126417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:33.126417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:33.126417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:33.126417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:33.126417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:33.126417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:33.126417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:33.126417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:33.126417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:33.127354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:33.127354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:33.127354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:33.127354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:33.127354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:33.127354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:33.127354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:33.127354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:33.127354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:33.127354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:33.127354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:33.127354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:33.127354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:33.127354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:33.127354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:33.127354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:33.128207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:33.128207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:33.128207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:33.128207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:33.128207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:33.128207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:33.128207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:33.128207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:33.128207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:33.128207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:33.128207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:04:33.128207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:33.128207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 220/444] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:04:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:33.128917 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8faec2-96d8-4ea1-baba-42f50c4a9b9e None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:33.155225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 225/445] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:33.156054 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72eb72-7d41-4055-8944-16b16cc76b60 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:33.229467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8faec2-96d8-4ea1-baba-42f50c4a9b9e None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:33.253636 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a72eb72-7d41-4055-8944-16b16cc76b60 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:33.263108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c26346-39bb-4f58-9181-ef10c6e4fcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:33.263745 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c26346-39bb-4f58-9181-ef10c6e4fcf8 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:33.264029 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c26346-39bb-4f58-9181-ef10c6e4fcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:33.264304 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c26346-39bb-4f58-9181-ef10c6e4fcf8 None admin] PATH_INFO: `/v3/groups/a51f0767088e4c889d3c33d8675952ec` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:33.265502 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c26346-39bb-4f58-9181-ef10c6e4fcf8 None admin] RBAC: Authorizing `identity:get_group(group_id=a51f0767088e4c889d3c33d8675952ec)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:33.266960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c26346-39bb-4f58-9181-ef10c6e4fcf8 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:33.269048 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 221/446] 10.222.0.45 () {56 vars in 1180 bytes} [Thu Aug 4 02:04:33 2022] GET /identity/v3/groups/a51f0767088e4c889d3c33d8675952ec => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:33.269971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c26346-39bb-4f58-9181-ef10c6e4fcf8 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:33.289278 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:33.293556 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd0e44-13fd-4144-8f5d-101ae2ca8186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:33.294178 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd0e44-13fd-4144-8f5d-101ae2ca8186 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:33.294456 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd0e44-13fd-4144-8f5d-101ae2ca8186 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:33.294730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd0e44-13fd-4144-8f5d-101ae2ca8186 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:33.295451 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd0e44-13fd-4144-8f5d-101ae2ca8186 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:33.295754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd0e44-13fd-4144-8f5d-101ae2ca8186 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:33.297159 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd0e44-13fd-4144-8f5d-101ae2ca8186 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:33.305764 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 226/447] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:33.306570 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd0e44-13fd-4144-8f5d-101ae2ca8186 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:33.335108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:33.369156 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c26346-39bb-4f58-9181-ef10c6e4fcf8 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:33.402190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f35d02-0444-45f4-a015-78b74bedb515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:33.402823 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f35d02-0444-45f4-a015-78b74bedb515 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:33.403104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f35d02-0444-45f4-a015-78b74bedb515 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:33.403381 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f35d02-0444-45f4-a015-78b74bedb515 None admin] PATH_INFO: `/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:33.404540 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f35d02-0444-45f4-a015-78b74bedb515 None admin] RBAC: Authorizing `identity:get_project(project_id=c872c66e5ba64576ab2eac8ce92c8b92)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:33.406030 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f35d02-0444-45f4-a015-78b74bedb515 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:33.406129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd0e44-13fd-4144-8f5d-101ae2ca8186 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:33.407678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 222/448] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:33 2022] GET /identity/v3/projects/c872c66e5ba64576ab2eac8ce92c8b92 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:33.408437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f35d02-0444-45f4-a015-78b74bedb515 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:33.445214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634cfb9-e0f0-41b1-9536-9c54004be6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:33.445838 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634cfb9-e0f0-41b1-9536-9c54004be6ef None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:33.446121 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634cfb9-e0f0-41b1-9536-9c54004be6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:33.446386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634cfb9-e0f0-41b1-9536-9c54004be6ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:33.447164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0634cfb9-e0f0-41b1-9536-9c54004be6ef None admin] RBAC: Adding query filter params (group.id=a51f0767088e4c889d3c33d8675952ec, role.id=8fc75c351a0349e294deeed1ff391c4c, scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:33.447467 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0634cfb9-e0f0-41b1-9536-9c54004be6ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:33.448896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0634cfb9-e0f0-41b1-9536-9c54004be6ef None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:33.455377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 227/449] 10.222.0.45 () {56 vars in 1376 bytes} [Thu Aug 4 02:04:33 2022] GET /identity/v3/role_assignments?group.id=a51f0767088e4c889d3c33d8675952ec&scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92&role.id=8fc75c351a0349e294deeed1ff391c4c => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:33.456195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634cfb9-e0f0-41b1-9536-9c54004be6ef None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:33.501126 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:33.519237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f35d02-0444-45f4-a015-78b74bedb515 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:33.557839 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f352778-708c-453f-b7fb-728ac0519c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:33.558463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f352778-708c-453f-b7fb-728ac0519c71 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:33.558738 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f352778-708c-453f-b7fb-728ac0519c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:33.559006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f352778-708c-453f-b7fb-728ac0519c71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:33.564423 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:33.574972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634cfb9-e0f0-41b1-9536-9c54004be6ef None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:33.615673 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8016f2-2685-4539-99a8-15ee5baf5af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:33.616647 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8016f2-2685-4539-99a8-15ee5baf5af7 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:33.616962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8016f2-2685-4539-99a8-15ee5baf5af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:33.617243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8016f2-2685-4539-99a8-15ee5baf5af7 None admin] PATH_INFO: `/v3/groups/a51f0767088e4c889d3c33d8675952ec` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:33.618449 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8016f2-2685-4539-99a8-15ee5baf5af7 None admin] RBAC: Authorizing `identity:get_group(group_id=a51f0767088e4c889d3c33d8675952ec)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:33.620705 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8016f2-2685-4539-99a8-15ee5baf5af7 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:33.622371 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 228/450] 10.222.0.45 () {56 vars in 1180 bytes} [Thu Aug 4 02:04:33 2022] GET /identity/v3/groups/a51f0767088e4c889d3c33d8675952ec => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:33.628501 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8016f2-2685-4539-99a8-15ee5baf5af7 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:33.642768 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f352778-708c-453f-b7fb-728ac0519c71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:33.647186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f352778-708c-453f-b7fb-728ac0519c71 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:33.753910 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:33.756339 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8016f2-2685-4539-99a8-15ee5baf5af7 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:33.772442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 223/451] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:33.777595 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f352778-708c-453f-b7fb-728ac0519c71 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:33.799279 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a78fe8-8012-46f5-9c45-8f63f1ff93d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:33.800288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a78fe8-8012-46f5-9c45-8f63f1ff93d6 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:33.801241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a78fe8-8012-46f5-9c45-8f63f1ff93d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:33.801241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a78fe8-8012-46f5-9c45-8f63f1ff93d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:33.801662 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a78fe8-8012-46f5-9c45-8f63f1ff93d6 None admin] RBAC: Adding query filter params (group.id=a51f0767088e4c889d3c33d8675952ec, role.id=80e5528d7c8742f78e29e64c1216176c, scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:33.803589 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a78fe8-8012-46f5-9c45-8f63f1ff93d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:33.803589 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a78fe8-8012-46f5-9c45-8f63f1ff93d6 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:33.809928 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 229/452] 10.222.0.45 () {56 vars in 1376 bytes} [Thu Aug 4 02:04:33 2022] GET /identity/v3/role_assignments?group.id=a51f0767088e4c889d3c33d8675952ec&scope.project.id=c872c66e5ba64576ab2eac8ce92c8b92&role.id=80e5528d7c8742f78e29e64c1216176c => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:33.884930 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f352778-708c-453f-b7fb-728ac0519c71 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:33.926946 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720496c-3bc9-4062-ab8a-837516061f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:33.927600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720496c-3bc9-4062-ab8a-837516061f3f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:33.927877 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720496c-3bc9-4062-ab8a-837516061f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:33.928149 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720496c-3bc9-4062-ab8a-837516061f3f None admin] PATH_INFO: `/v3/projects/457cd62a685649778d5130f3937b1bcf` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:33.929390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720496c-3bc9-4062-ab8a-837516061f3f None admin] RBAC: Authorizing `identity:get_project(project_id=457cd62a685649778d5130f3937b1bcf)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:33.930863 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720496c-3bc9-4062-ab8a-837516061f3f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:33.932597 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 224/453] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:04:33 2022] GET /identity/v3/projects/457cd62a685649778d5130f3937b1bcf => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:33.938689 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a78fe8-8012-46f5-9c45-8f63f1ff93d6 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:33.969906 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:34.055539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a78fe8-8012-46f5-9c45-8f63f1ff93d6 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:34.091132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b2a94f-4c6b-413c-a0da-0192040fee66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:34.091891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2a94f-4c6b-413c-a0da-0192040fee66 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:34.092179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2a94f-4c6b-413c-a0da-0192040fee66 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:34.092567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b2a94f-4c6b-413c-a0da-0192040fee66 None admin] PATH_INFO: `/v3/groups/223a249772584fa892f68c98d61a85e3` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:34.094621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2a94f-4c6b-413c-a0da-0192040fee66 None admin] RBAC: Authorizing `identity:get_group(group_id=223a249772584fa892f68c98d61a85e3)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:34.096073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b2a94f-4c6b-413c-a0da-0192040fee66 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:34.097760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 230/454] 10.222.0.45 () {56 vars in 1180 bytes} [Thu Aug 4 02:04:33 2022] GET /identity/v3/groups/223a249772584fa892f68c98d61a85e3 => generated 229 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:34.103818 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720496c-3bc9-4062-ab8a-837516061f3f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:34.161123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:34.207203 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720496c-3bc9-4062-ab8a-837516061f3f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:34.239742 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89488d45-085c-418d-9547-1e47925c7238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:34.240341 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89488d45-085c-418d-9547-1e47925c7238 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:34.240636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89488d45-085c-418d-9547-1e47925c7238 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:34.240914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89488d45-085c-418d-9547-1e47925c7238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:34.241698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89488d45-085c-418d-9547-1e47925c7238 None admin] RBAC: Adding query filter params (group.id=223a249772584fa892f68c98d61a85e3, role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=457cd62a685649778d5130f3937b1bcf) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:34.242009 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89488d45-085c-418d-9547-1e47925c7238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:34.243428 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89488d45-085c-418d-9547-1e47925c7238 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:34.249047 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 225/455] 10.222.0.45 () {56 vars in 1376 bytes} [Thu Aug 4 02:04:34 2022] GET /identity/v3/role_assignments?group.id=223a249772584fa892f68c98d61a85e3&scope.project.id=457cd62a685649778d5130f3937b1bcf&role.id=35205394c8af4c52a61b12f92ba16644 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:36.816314 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead0d10-79b1-4529-9f97-26dbd27bc436 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:36.816843 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead0d10-79b1-4529-9f97-26dbd27bc436 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:36.816915 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aead0d10-79b1-4529-9f97-26dbd27bc436 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:36.818491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 231/456] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:36.823458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87574ca4-b40a-4910-8e59-fbe3d9c95dcf None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:36.824329 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87574ca4-b40a-4910-8e59-fbe3d9c95dcf None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:36.824844 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87574ca4-b40a-4910-8e59-fbe3d9c95dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:36.849608 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-87574ca4-b40a-4910-8e59-fbe3d9c95dcf None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:36.851147 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-87574ca4-b40a-4910-8e59-fbe3d9c95dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:36.942194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a39968-dcf0-4cf1-92f9-1c267cb54a13 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:36.942986 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a39968-dcf0-4cf1-92f9-1c267cb54a13 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:36.943146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 226/457] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:36.943608 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a39968-dcf0-4cf1-92f9-1c267cb54a13 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:36.944980 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 232/458] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:36.962498 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff734e-2059-4cb7-a7b3-d52a7015eda9 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:36.962812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff734e-2059-4cb7-a7b3-d52a7015eda9 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:36.963057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff734e-2059-4cb7-a7b3-d52a7015eda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:36.966972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a39968-dcf0-4cf1-92f9-1c267cb54a13 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:36.987438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-d4ff734e-2059-4cb7-a7b3-d52a7015eda9 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:36.988649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d4ff734e-2059-4cb7-a7b3-d52a7015eda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:37.073406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:37.105759 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 227/459] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:37.107775 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615326a0-00c7-422f-b0e8-47e8b5142416 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:37.108223 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615326a0-00c7-422f-b0e8-47e8b5142416 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:37.108960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615326a0-00c7-422f-b0e8-47e8b5142416 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:37.110966 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 228/460] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:37.117286 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc79ed2-5405-4233-98ae-6d8125a6ed3f None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:37.117723 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc79ed2-5405-4233-98ae-6d8125a6ed3f None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:37.118125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc79ed2-5405-4233-98ae-6d8125a6ed3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:37.129974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:37.141844 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-8dc79ed2-5405-4233-98ae-6d8125a6ed3f None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:37.143996 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-8dc79ed2-5405-4233-98ae-6d8125a6ed3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:37.146409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a39968-dcf0-4cf1-92f9-1c267cb54a13 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:37.193286 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e28756-dbab-4f87-ad2e-02c97897d90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:37.194231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e28756-dbab-4f87-ad2e-02c97897d90b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:37.194702 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e28756-dbab-4f87-ad2e-02c97897d90b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:37.195474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e28756-dbab-4f87-ad2e-02c97897d90b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:37.213031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e28756-dbab-4f87-ad2e-02c97897d90b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:37.215091 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e28756-dbab-4f87-ad2e-02c97897d90b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:37.234803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-95e28756-dbab-4f87-ad2e-02c97897d90b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:37.234803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:37.234803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:37.234803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:37.234803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:37.234803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:37.234803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:37.234803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:37.234803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:37.234803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:37.234803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:37.234803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:37.234803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:37.234803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:37.234803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:37.235706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:37.235706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:37.235706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:37.235706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:37.235706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:37.235706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:37.235706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:37.235706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:37.235706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:37.235706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:37.235706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:37.235706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:37.235706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:37.235706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:37.235706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:37.235706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:37.235706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:37.236524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:37.236524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:37.236524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:37.236524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:37.243245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 229/462] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:04:37.243245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc79ed2-5405-4233-98ae-6d8125a6ed3f None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:37.243765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 233/462] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:37.246344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e28756-dbab-4f87-ad2e-02c97897d90b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:37.354768 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc79ed2-5405-4233-98ae-6d8125a6ed3f None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:37.366770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:37.368492 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e28756-dbab-4f87-ad2e-02c97897d90b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:37.395522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98e383-2905-4e79-9872-b1427fa6b7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:37.396475 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98e383-2905-4e79-9872-b1427fa6b7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:37.396927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98e383-2905-4e79-9872-b1427fa6b7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:37.397370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a98e383-2905-4e79-9872-b1427fa6b7b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:37.407453 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87a95f-53f8-4c18-869f-8c89eb1f3551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:37.407809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98e383-2905-4e79-9872-b1427fa6b7b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:37.408073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87a95f-53f8-4c18-869f-8c89eb1f3551 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:37.408352 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87a95f-53f8-4c18-869f-8c89eb1f3551 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:37.408651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87a95f-53f8-4c18-869f-8c89eb1f3551 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:37.409318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a98e383-2905-4e79-9872-b1427fa6b7b1 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:37.409420 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87a95f-53f8-4c18-869f-8c89eb1f3551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:37.409689 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87a95f-53f8-4c18-869f-8c89eb1f3551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:37.411235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87a95f-53f8-4c18-869f-8c89eb1f3551 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:37.423726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-2a98e383-2905-4e79-9872-b1427fa6b7b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:37.423726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:37.423726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:37.423726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:37.423726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:37.423726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:37.423726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:37.423726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:37.423726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:37.423726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:37.423726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:37.423726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:37.423726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:37.423726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:37.423726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:37.424571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:37.424571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:37.424571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:37.424571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:37.424571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:37.424571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:37.424571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:37.424571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:37.424571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:37.424571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:37.424571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:37.424571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:37.424571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:37.424571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:37.424571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:37.424571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:37.424571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:37.426010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:37.426010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:37.426010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:37.426010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:37.426010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 230/463] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:37.426010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98e383-2905-4e79-9872-b1427fa6b7b1 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:37.433636 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 234/464] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:37.434780 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a703e6-c0ed-419f-ac1f-2a225138b7a9 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:37.435064 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a703e6-c0ed-419f-ac1f-2a225138b7a9 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:37.435339 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a703e6-c0ed-419f-ac1f-2a225138b7a9 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:37.437118 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 235/465] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:37.438460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a703e6-c0ed-419f-ac1f-2a225138b7a9 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:37.442765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:37.543621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a98e383-2905-4e79-9872-b1427fa6b7b1 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:37.562036 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a703e6-c0ed-419f-ac1f-2a225138b7a9 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:37.578454 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a58652-5345-42d9-a04c-22c8e52e3344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:37.579090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a58652-5345-42d9-a04c-22c8e52e3344 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:37.579369 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a58652-5345-42d9-a04c-22c8e52e3344 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:37.579640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a58652-5345-42d9-a04c-22c8e52e3344 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:37.590093 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a58652-5345-42d9-a04c-22c8e52e3344 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:37.591535 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a58652-5345-42d9-a04c-22c8e52e3344 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:37.602098 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a21f30-64e7-40dd-b054-9408697d986a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:37.602727 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a21f30-64e7-40dd-b054-9408697d986a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:37.603008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a21f30-64e7-40dd-b054-9408697d986a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:37.603280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a21f30-64e7-40dd-b054-9408697d986a None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:37.603975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a21f30-64e7-40dd-b054-9408697d986a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:37.604278 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a21f30-64e7-40dd-b054-9408697d986a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:37.605693 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a21f30-64e7-40dd-b054-9408697d986a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:37.606564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-e8a58652-5345-42d9-a04c-22c8e52e3344 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:37.606564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:37.606564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:37.606564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:37.606564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:37.606564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:37.606564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:37.606564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:37.606564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:37.606564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:37.606564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:37.606564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:37.606564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:37.606564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:37.606564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:37.607384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:37.607384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:37.607384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:37.607384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:37.607384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:37.607384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:37.607384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:37.607384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:37.607384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:37.607384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:37.607384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:37.607384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:37.607384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:37.607384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:37.607384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:37.607384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:37.607384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:37.608228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:37.608228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:37.608228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:37.608228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:37.608228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 231/466] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:37.614579 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a58652-5345-42d9-a04c-22c8e52e3344 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:37.619315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 236/467] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:37.620343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d257ed-ebd9-4fce-8a12-bc80ff2f6bad None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:37.620650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d257ed-ebd9-4fce-8a12-bc80ff2f6bad None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:37.620927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d257ed-ebd9-4fce-8a12-bc80ff2f6bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:37.631765 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:37.639863 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-b4d257ed-ebd9-4fce-8a12-bc80ff2f6bad None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:37.641099 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b4d257ed-ebd9-4fce-8a12-bc80ff2f6bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:37.705671 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:37.720679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 237/468] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:37.720679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d257ed-ebd9-4fce-8a12-bc80ff2f6bad None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:37.739824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a58652-5345-42d9-a04c-22c8e52e3344 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:37.775552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49a523-f246-456b-ad1e-098a87d4d61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:37.776174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49a523-f246-456b-ad1e-098a87d4d61f None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:37.776454 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49a523-f246-456b-ad1e-098a87d4d61f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:37.776758 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49a523-f246-456b-ad1e-098a87d4d61f None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:37.777525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49a523-f246-456b-ad1e-098a87d4d61f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:37.778989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49a523-f246-456b-ad1e-098a87d4d61f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:37.801800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 232/469] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:04:37 2022] POST /identity/v3/users => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:04:37.802514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49a523-f246-456b-ad1e-098a87d4d61f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:37.846141 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d257ed-ebd9-4fce-8a12-bc80ff2f6bad None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:37.895528 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275e266-9cbd-4c24-9269-8b529024cb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:37.896814 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:37.899107 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275e266-9cbd-4c24-9269-8b529024cb2d None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:37.899635 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275e266-9cbd-4c24-9269-8b529024cb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:37.900465 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1275e266-9cbd-4c24-9269-8b529024cb2d None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:37.903906 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275e266-9cbd-4c24-9269-8b529024cb2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:37.904524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275e266-9cbd-4c24-9269-8b529024cb2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:37.906288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1275e266-9cbd-4c24-9269-8b529024cb2d None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:37.922768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 238/470] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:37.924514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275e266-9cbd-4c24-9269-8b529024cb2d None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:37.941797 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49a523-f246-456b-ad1e-098a87d4d61f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:37.989386 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52cd43-191d-4c1e-98d8-d9afb5b17c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:37.990974 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52cd43-191d-4c1e-98d8-d9afb5b17c40 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:37.991470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52cd43-191d-4c1e-98d8-d9afb5b17c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:37.991751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52cd43-191d-4c1e-98d8-d9afb5b17c40 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:37.992541 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52cd43-191d-4c1e-98d8-d9afb5b17c40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:37.994700 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52cd43-191d-4c1e-98d8-d9afb5b17c40 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:38.010166 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:38.020073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 233/471] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:04:37 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:04:38.020936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52cd43-191d-4c1e-98d8-d9afb5b17c40 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:38.079990 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1275e266-9cbd-4c24-9269-8b529024cb2d None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:38.117192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293f099-6953-4959-85a0-1704694cdbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:38.117840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293f099-6953-4959-85a0-1704694cdbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:38.118127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293f099-6953-4959-85a0-1704694cdbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:38.118395 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293f099-6953-4959-85a0-1704694cdbd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:38.131122 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293f099-6953-4959-85a0-1704694cdbd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:38.132596 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293f099-6953-4959-85a0-1704694cdbd3 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:38.152013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-f293f099-6953-4959-85a0-1704694cdbd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:38.152013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:38.152013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:38.152013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:38.152013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:38.152013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:38.152013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:38.152013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:38.152013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:38.152013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:38.152013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:38.152013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:38.152013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:38.152013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:38.152013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:38.153009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:38.153009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:38.153009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:38.153009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:38.153009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:38.153009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:38.153009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:38.153009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:38.153009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:38.153009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:38.153009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:38.153009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:38.153009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:38.153009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:38.153009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:38.153009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:38.153009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:38.153834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:38.153834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:38.153834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:38.153834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:38.153834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 239/472] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:38.154867 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31ec92-c219-4a27-bd4f-99d9a367dfd4 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:38.155149 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31ec92-c219-4a27-bd4f-99d9a367dfd4 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:38.155424 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f31ec92-c219-4a27-bd4f-99d9a367dfd4 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:38.157012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 240/473] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:38.158849 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31ec92-c219-4a27-bd4f-99d9a367dfd4 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:38.165275 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52cd43-191d-4c1e-98d8-d9afb5b17c40 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:38.192644 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:38.208587 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d46c0c-0b24-4d56-862b-55711a024e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:38.209654 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d46c0c-0b24-4d56-862b-55711a024e82 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:38.210140 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d46c0c-0b24-4d56-862b-55711a024e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:38.210613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d46c0c-0b24-4d56-862b-55711a024e82 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:38.211526 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d46c0c-0b24-4d56-862b-55711a024e82 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:38.213315 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d46c0c-0b24-4d56-862b-55711a024e82 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:38.243130 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 234/474] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:04:38 2022] POST /identity/v3/users => generated 248 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:04:38.244703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7359fc-dd6a-4681-aadd-a9f141d7110c None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:38.245160 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7359fc-dd6a-4681-aadd-a9f141d7110c None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:38.245605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7359fc-dd6a-4681-aadd-a9f141d7110c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:38.268273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-ba7359fc-dd6a-4681-aadd-a9f141d7110c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:38.270206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ba7359fc-dd6a-4681-aadd-a9f141d7110c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:38.306219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f31ec92-c219-4a27-bd4f-99d9a367dfd4 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:38.316244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:38.369094 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70f620-2e37-4c25-b9d0-4ebf558096a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:38.370207 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70f620-2e37-4c25-b9d0-4ebf558096a1 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:38.370633 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70f620-2e37-4c25-b9d0-4ebf558096a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:38.371050 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70f620-2e37-4c25-b9d0-4ebf558096a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:38.371465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 235/475] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:38.372085 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70f620-2e37-4c25-b9d0-4ebf558096a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:38.372395 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70f620-2e37-4c25-b9d0-4ebf558096a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:38.374107 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a70f620-2e37-4c25-b9d0-4ebf558096a1 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:38.389349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 241/476] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:38.395764 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7359fc-dd6a-4681-aadd-a9f141d7110c None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:38.408237 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70f620-2e37-4c25-b9d0-4ebf558096a1 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:38.517748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7359fc-dd6a-4681-aadd-a9f141d7110c None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:38.519228 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70f620-2e37-4c25-b9d0-4ebf558096a1 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:38.530457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:38.556661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d1498-9641-4c38-bbbc-5655658db13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:38.557532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d1498-9641-4c38-bbbc-5655658db13f None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:38.557995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d1498-9641-4c38-bbbc-5655658db13f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:38.558363 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68d1498-9641-4c38-bbbc-5655658db13f None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:38.559391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d1498-9641-4c38-bbbc-5655658db13f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:38.560726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa8431-9eb5-4bc7-8595-e16a959727ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:38.561153 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68d1498-9641-4c38-bbbc-5655658db13f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:38.561936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa8431-9eb5-4bc7-8595-e16a959727ed None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:38.562346 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa8431-9eb5-4bc7-8595-e16a959727ed None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:38.562747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa8431-9eb5-4bc7-8595-e16a959727ed None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:38.563558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa8431-9eb5-4bc7-8595-e16a959727ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:38.565585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aa8431-9eb5-4bc7-8595-e16a959727ed None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:38.571305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.sql.core [None req-c68d1498-9641-4c38-bbbc-5655658db13f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 02:04:38.571305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 02:04:38.571305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [parameters: {'id': '215edaceb4c64a7e841394d94ee19c25', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 02:04:38.571305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80672) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 02:04:38.590681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-c68d1498-9641-4c38-bbbc-5655658db13f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 02:04:38.590681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:38.590681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 02:04:38.590681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 02:04:38.590681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 02:04:38.590681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 02:04:38.590681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 02:04:38.590681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 02:04:38.590681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 02:04:38.590681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application conn.query(q) Aug 04 02:04:38.590681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 02:04:38.590681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 02:04:38.590681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 02:04:38.590681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application result.read() Aug 04 02:04:38.590681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return ref.to_dict() Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 02:04:38.591821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application next(self.gen) Aug 04 02:04:38.592996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 02:04:38.592996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application yield resource Aug 04 02:04:38.592996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 02:04:38.592996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application next(self.gen) Aug 04 02:04:38.592996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 02:04:38.592996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 02:04:38.592996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 02:04:38.592996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 02:04:38.592996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 02:04:38.592996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise self.value Aug 04 02:04:38.592996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 02:04:38.592996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 02:04:38.592996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 02:04:38.592996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application session.commit() Aug 04 02:04:38.592996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 02:04:38.592996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 02:04:38.592996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 02:04:38.594046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 02:04:38.594046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 02:04:38.594046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self.session.flush() Aug 04 02:04:38.594046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 02:04:38.594046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 02:04:38.594046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 02:04:38.594046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 02:04:38.594046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 02:04:38.594046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application compat.raise_( Aug 04 02:04:38.594046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 02:04:38.594046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception Aug 04 02:04:38.594046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 02:04:38.594046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 02:04:38.594046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 02:04:38.594046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 02:04:38.594046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 02:04:38.594046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 02:04:38.595100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 02:04:38.595100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 02:04:38.595100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 02:04:38.595100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 02:04:38.595100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 02:04:38.595100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 02:04:38.595100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 02:04:38.595100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 02:04:38.595100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 02:04:38.595100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 02:04:38.595100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 02:04:38.595100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 02:04:38.595100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 02:04:38.595100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 02:04:38.595100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 02:04:38.595100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception Aug 04 02:04:38.596093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 02:04:38.596093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 02:04:38.596093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 02:04:38.596093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 02:04:38.596093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 02:04:38.596093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 02:04:38.596093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 02:04:38.596093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application conn.query(q) Aug 04 02:04:38.596093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 02:04:38.596093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 02:04:38.596093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 02:04:38.596093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application result.read() Aug 04 02:04:38.596093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 02:04:38.596093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 02:04:38.596093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 02:04:38.596093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 02:04:38.597692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 02:04:38.597692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 02:04:38.597692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 02:04:38.597692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 02:04:38.597692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 04 02:04:38.597692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 04 02:04:38.597692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [parameters: {'id': '215edaceb4c64a7e841394d94ee19c25', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 04 02:04:38.597692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 02:04:38.597692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:38.597692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:04:38.597692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:38.597692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:38.597692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:38.597692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:38.597692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:38.597692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:38.598720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:38.598720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:38.598720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:38.598720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:38.598720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:38.598720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:38.598720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 04 02:04:38.598720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 04 02:04:38.598720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:38.598720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:38.598720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 04 02:04:38.598720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 04 02:04:38.598720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 02:04:38.598720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 02:04:38.598720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 04 02:04:38.598720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:38.604007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 242/477] 10.222.0.45 () {60 vars in 1131 bytes} [Thu Aug 4 02:04:38 2022] POST /identity/v3/roles => generated 144 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:38.605095 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 236/478] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:04:38 2022] POST /identity/v3/users => generated 249 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:04:38.607178 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d1498-9641-4c38-bbbc-5655658db13f None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:38.644009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:38.727717 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68d1498-9641-4c38-bbbc-5655658db13f None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:38.767779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc87384-2bc7-4a31-9afc-446fc21cbe77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:38.768491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc87384-2bc7-4a31-9afc-446fc21cbe77 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:38.768809 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc87384-2bc7-4a31-9afc-446fc21cbe77 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:38.769085 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc87384-2bc7-4a31-9afc-446fc21cbe77 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:38.782645 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc87384-2bc7-4a31-9afc-446fc21cbe77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:38.788077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-1dc87384-2bc7-4a31-9afc-446fc21cbe77 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:38.788077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:38.788077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:38.788077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:38.788077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:38.788077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:38.788077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:38.788077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:38.788077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:38.788077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:38.788077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:38.788077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:38.788077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:38.788077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:38.788077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:38.788077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:38.788970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:38.788970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:38.788970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:38.788970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:38.788970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:38.788970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:38.788970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:38.788970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:38.788970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:38.788970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:38.788970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:38.788970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:38.788970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:38.788970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:38.788970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:38.788970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:38.789758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:38.789758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:38.789758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:38.789758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:38.789758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:38.789758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:38.789758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:38.789758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:38.789758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:38.789758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:38.789758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 04 02:04:38.789758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:38.789758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 243/479] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:04:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:38.794926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa8431-9eb5-4bc7-8595-e16a959727ed None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:38.914358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa8431-9eb5-4bc7-8595-e16a959727ed None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:38.953957 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30223137-f0fb-4ac5-8116-05e082dde4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:38.954592 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30223137-f0fb-4ac5-8116-05e082dde4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:38.954927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30223137-f0fb-4ac5-8116-05e082dde4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:38.955199 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30223137-f0fb-4ac5-8116-05e082dde4c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:38.955999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30223137-f0fb-4ac5-8116-05e082dde4c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:38.956311 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30223137-f0fb-4ac5-8116-05e082dde4c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:38.957789 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30223137-f0fb-4ac5-8116-05e082dde4c5 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:38.967494 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 237/480] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:40.292168 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c3f83-01e3-47dd-a341-76a6b24a160a None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:40.292735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c3f83-01e3-47dd-a341-76a6b24a160a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:40.292735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c3f83-01e3-47dd-a341-76a6b24a160a None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:40.294360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 244/481] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:40.299183 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab984ed-2b63-4c9c-909d-e5cdb46d7beb None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:40.299578 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab984ed-2b63-4c9c-909d-e5cdb46d7beb None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:40.300295 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab984ed-2b63-4c9c-909d-e5cdb46d7beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:40.319696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-5ab984ed-2b63-4c9c-909d-e5cdb46d7beb None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:40.325308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-5ab984ed-2b63-4c9c-909d-e5cdb46d7beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:40.349410 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:40.411593 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 238/482] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:40.448424 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c3f83-01e3-47dd-a341-76a6b24a160a None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:40.571547 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:40.575591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c3433-2cdd-40f4-84ae-6372b3b13871 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:40.575591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c3433-2cdd-40f4-84ae-6372b3b13871 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:40.575591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c3433-2cdd-40f4-84ae-6372b3b13871 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:40.576208 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 239/483] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:04:40.583585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c515d-5c97-4ca0-bf48-5910f28f19ca None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:40.583867 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c515d-5c97-4ca0-bf48-5910f28f19ca None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:40.584140 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c515d-5c97-4ca0-bf48-5910f28f19ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:40.591065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c3f83-01e3-47dd-a341-76a6b24a160a None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:40.606506 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-b41c515d-5c97-4ca0-bf48-5910f28f19ca None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:40.609918 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-b41c515d-5c97-4ca0-bf48-5910f28f19ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:40.665625 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6735d5-0199-48cf-9612-44731d7d0ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:40.666605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6735d5-0199-48cf-9612-44731d7d0ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:40.667079 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6735d5-0199-48cf-9612-44731d7d0ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:40.667571 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6735d5-0199-48cf-9612-44731d7d0ca1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:40.689026 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6735d5-0199-48cf-9612-44731d7d0ca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:40.697163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-4f6735d5-0199-48cf-9612-44731d7d0ca1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:40.697163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:40.697163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:40.697163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:40.697163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:40.697163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:40.697163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:40.697163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:40.697163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:40.697163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:40.697163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:40.697163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:40.697163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:40.697163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:40.697163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:40.697163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:40.698139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:40.698139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:40.698139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:40.698139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:40.698139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:40.698139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:40.698139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:40.698139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:40.698139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:40.698139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:40.698139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:40.698139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:40.698139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:40.698139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:40.698139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:40.698139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:40.698920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:40.698920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:40.698920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:40.698920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:40.698920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:40.698920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:40.698920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:40.698920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:40.698920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:40.698920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:40.698920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:40.698920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:40.703600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 245/484] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:40.709007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6735d5-0199-48cf-9612-44731d7d0ca1 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:40.754892 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 240/485] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:04:40.778147 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c515d-5c97-4ca0-bf48-5910f28f19ca None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:40.867416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6735d5-0199-48cf-9612-44731d7d0ca1 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:40.904066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b0cd0-e1bd-45bb-8589-fad594b02966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:40.904734 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b0cd0-e1bd-45bb-8589-fad594b02966 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:40.905014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b0cd0-e1bd-45bb-8589-fad594b02966 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:40.905287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b0cd0-e1bd-45bb-8589-fad594b02966 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:40.906010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b0cd0-e1bd-45bb-8589-fad594b02966 None admin] RBAC: Adding query filter params (name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:40.906313 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b0cd0-e1bd-45bb-8589-fad594b02966 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:40.907701 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b0cd0-e1bd-45bb-8589-fad594b02966 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:40.917029 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 246/486] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:04:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:40.922069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b0cd0-e1bd-45bb-8589-fad594b02966 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:40.943157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:40.945163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:40.982602 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c515d-5c97-4ca0-bf48-5910f28f19ca None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:41.032131 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b0cd0-e1bd-45bb-8589-fad594b02966 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:41.050779 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e26e0a-5332-4a22-8c2e-e74b728eef69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:41.051442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e26e0a-5332-4a22-8c2e-e74b728eef69 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:41.051724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e26e0a-5332-4a22-8c2e-e74b728eef69 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:41.051998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e26e0a-5332-4a22-8c2e-e74b728eef69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:41.070175 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6059e20d-40b4-4342-b286-a22a1a628baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:41.070840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059e20d-40b4-4342-b286-a22a1a628baf None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:41.071218 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059e20d-40b4-4342-b286-a22a1a628baf None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:41.071494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6059e20d-40b4-4342-b286-a22a1a628baf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:41.074211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e26e0a-5332-4a22-8c2e-e74b728eef69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:41.082657 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059e20d-40b4-4342-b286-a22a1a628baf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:41.084244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6059e20d-40b4-4342-b286-a22a1a628baf None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:41.087373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-d8e26e0a-5332-4a22-8c2e-e74b728eef69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:41.087373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:41.087373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:41.087373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:41.087373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:41.087373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:41.087373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:41.087373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:41.087373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:41.087373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:41.087373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:41.087373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:41.087373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:41.087373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:41.087373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:41.087373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:41.088247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:41.088247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:41.088247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:41.088247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:41.088247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:41.088247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:41.088247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:41.088247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:41.088247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:41.088247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:41.088247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:41.088247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:41.088247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:41.088247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:41.088247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:41.088247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:41.089091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:41.089091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:41.089091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:41.089091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:41.089091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:41.089091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:41.089091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:41.089091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:41.089091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:41.089091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:41.089091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:41.089091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:41.089091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 241/487] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:41.099471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec906afc-b5c9-4555-ba28-731de2a281f5 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:41.099754 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec906afc-b5c9-4555-ba28-731de2a281f5 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:41.100032 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec906afc-b5c9-4555-ba28-731de2a281f5 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:41.104234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 242/488] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:41.105120 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec906afc-b5c9-4555-ba28-731de2a281f5 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:41.109532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-6059e20d-40b4-4342-b286-a22a1a628baf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:41.109532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:41.109532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:41.109532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:41.109532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:41.109532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:41.109532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:41.109532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:41.109532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:41.109532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:41.109532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:41.109532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:41.109532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:41.109532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:41.109532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:41.110377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:41.110377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:41.110377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:41.110377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:41.110377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:41.110377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:41.110377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:41.110377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:41.110377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:41.110377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:41.110377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:41.110377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:41.110377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:41.110377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:41.110377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:41.110377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:41.110377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:41.111198 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:41.111198 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:41.111198 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:41.111198 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:41.113872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 247/489] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:41.115310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0504e8-9ac9-414f-9be2-6cb69efd1717 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:41.115734 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0504e8-9ac9-414f-9be2-6cb69efd1717 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:41.116144 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0504e8-9ac9-414f-9be2-6cb69efd1717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:41.135857 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-9f0504e8-9ac9-414f-9be2-6cb69efd1717 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:41.137346 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9f0504e8-9ac9-414f-9be2-6cb69efd1717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:41.202862 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:41.213685 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 248/490] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:41.214798 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0504e8-9ac9-414f-9be2-6cb69efd1717 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:41.223386 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec906afc-b5c9-4555-ba28-731de2a281f5 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:41.273989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa1284-1634-4663-9735-69d74305c382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:41.274688 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa1284-1634-4663-9735-69d74305c382 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:41.274938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa1284-1634-4663-9735-69d74305c382 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:41.275220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa1284-1634-4663-9735-69d74305c382 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:41.275956 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa1284-1634-4663-9735-69d74305c382 None admin] RBAC: Adding query filter params (name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:41.276259 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa1284-1634-4663-9735-69d74305c382 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:41.277697 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa1284-1634-4663-9735-69d74305c382 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:41.291448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 243/491] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:04:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:41.291448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e1f38-e3f7-4b54-ac26-64976244dab8 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:41.291448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e1f38-e3f7-4b54-ac26-64976244dab8 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:41.291448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e1f38-e3f7-4b54-ac26-64976244dab8 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:41.291448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 244/492] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:41.291448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e1f38-e3f7-4b54-ac26-64976244dab8 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:41.310692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:41.334116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0504e8-9ac9-414f-9be2-6cb69efd1717 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:41.370964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4452cb8-8522-4caa-ab83-234f032bffb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:41.371568 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4452cb8-8522-4caa-ab83-234f032bffb0 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:41.371851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4452cb8-8522-4caa-ab83-234f032bffb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:41.372184 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4452cb8-8522-4caa-ab83-234f032bffb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:41.372952 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4452cb8-8522-4caa-ab83-234f032bffb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:41.373382 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4452cb8-8522-4caa-ab83-234f032bffb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:41.374814 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4452cb8-8522-4caa-ab83-234f032bffb0 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:41.389161 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 249/493] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:41.389947 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4452cb8-8522-4caa-ab83-234f032bffb0 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:41.410429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e1f38-e3f7-4b54-ac26-64976244dab8 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:41.449175 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:41.451662 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749b059-52e4-4291-973b-9e1c671648f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:41.452691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749b059-52e4-4291-973b-9e1c671648f9 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:41.453215 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749b059-52e4-4291-973b-9e1c671648f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:41.453931 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749b059-52e4-4291-973b-9e1c671648f9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:41.466122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749b059-52e4-4291-973b-9e1c671648f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:41.472250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-6749b059-52e4-4291-973b-9e1c671648f9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:41.472250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:41.472250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:41.472250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:41.472250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:41.472250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:41.472250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:41.472250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:41.472250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:41.472250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:41.472250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:41.472250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:41.472250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:41.472250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:41.472250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:41.472250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:41.473306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:41.473306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:41.473306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:41.473306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:41.473306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:41.473306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:41.473306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:41.473306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:41.473306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:41.473306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:41.473306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:41.473306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:41.473306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:41.473306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:41.473306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:41.473306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:41.474308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:41.474308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:41.474308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:41.474308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:41.474308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:41.474308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:41.474308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:41.474308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:41.474308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:41.474308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:41.474308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:41.474308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:41.476267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 245/494] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:41.477801 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82767bf5-52a9-4257-b78a-85fa993c5cbf None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:41.478086 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82767bf5-52a9-4257-b78a-85fa993c5cbf None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:41.478344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82767bf5-52a9-4257-b78a-85fa993c5cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:41.496809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-82767bf5-52a9-4257-b78a-85fa993c5cbf None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:41.498008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-82767bf5-52a9-4257-b78a-85fa993c5cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:41.504347 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4452cb8-8522-4caa-ab83-234f032bffb0 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:41.541301 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283912a4-554d-4574-b6c9-948a8a87f5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:41.542043 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283912a4-554d-4574-b6c9-948a8a87f5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:41.542222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283912a4-554d-4574-b6c9-948a8a87f5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:41.542485 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283912a4-554d-4574-b6c9-948a8a87f5d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:41.553548 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283912a4-554d-4574-b6c9-948a8a87f5d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:41.555054 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283912a4-554d-4574-b6c9-948a8a87f5d8 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:41.562959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:41.568319 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 246/495] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:41.572895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82767bf5-52a9-4257-b78a-85fa993c5cbf None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:41.572985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-283912a4-554d-4574-b6c9-948a8a87f5d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:41.572985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:41.572985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:41.572985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:41.572985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:41.572985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:41.572985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:41.572985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:41.572985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:41.572985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:41.572985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:41.572985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:41.572985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:41.572985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:41.572985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:41.573773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:41.573773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:41.573773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:41.573773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:41.573773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:41.573773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:41.573773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:41.573773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:41.573773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:41.573773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:41.573773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:41.573773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:41.573773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:41.573773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:41.573773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:41.573773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:41.573773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:41.574840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:41.574840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:41.574840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:41.574840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:41.577684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 250/496] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:41.578514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283912a4-554d-4574-b6c9-948a8a87f5d8 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:41.683200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82767bf5-52a9-4257-b78a-85fa993c5cbf None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:41.697730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283912a4-554d-4574-b6c9-948a8a87f5d8 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:41.705851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:41.729701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff5864-724b-402e-8edb-4a5a7bf4b8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:41.730334 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff5864-724b-402e-8edb-4a5a7bf4b8cb None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:41.730615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff5864-724b-402e-8edb-4a5a7bf4b8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:41.730889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff5864-724b-402e-8edb-4a5a7bf4b8cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:41.733933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4053d9-5a66-46a1-9bab-5a5c81cf4349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:41.734478 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4053d9-5a66-46a1-9bab-5a5c81cf4349 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:41.734753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4053d9-5a66-46a1-9bab-5a5c81cf4349 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:41.735019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4053d9-5a66-46a1-9bab-5a5c81cf4349 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:41.735726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4053d9-5a66-46a1-9bab-5a5c81cf4349 None admin] RBAC: Adding query filter params (name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:41.736024 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4053d9-5a66-46a1-9bab-5a5c81cf4349 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:41.737436 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4053d9-5a66-46a1-9bab-5a5c81cf4349 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:41.738298 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff5864-724b-402e-8edb-4a5a7bf4b8cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:41.739744 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ff5864-724b-402e-8edb-4a5a7bf4b8cb None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:41.745447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 251/497] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:04:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:41.746236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4053d9-5a66-46a1-9bab-5a5c81cf4349 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:41.752938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-d3ff5864-724b-402e-8edb-4a5a7bf4b8cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:04:41.752938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:41.752938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:41.752938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:41.752938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:41.752938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:41.752938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:41.752938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:41.752938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:41.752938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:41.752938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:41.752938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:41.752938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:04:41.752938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:04:41.752938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:04:41.752938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:41.753883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:41.754810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:41.754810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:41.754810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:04:41.754810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:04:41.754810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:04:41.754810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:04:41.754810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:04:41.754810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:04:41.754810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:04:41.754810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:41.754810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 247/498] 10.222.0.45 () {56 vars in 1093 bytes} [Thu Aug 4 02:04:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:41.755467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff5864-724b-402e-8edb-4a5a7bf4b8cb None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:41.814076 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:41.845223 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4053d9-5a66-46a1-9bab-5a5c81cf4349 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:41.856999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ff5864-724b-402e-8edb-4a5a7bf4b8cb None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:41.878880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd86fb9-cbc1-4c76-8d1a-3ea92c50c66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:41.879486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd86fb9-cbc1-4c76-8d1a-3ea92c50c66a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:41.879763 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd86fb9-cbc1-4c76-8d1a-3ea92c50c66a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:41.880036 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd86fb9-cbc1-4c76-8d1a-3ea92c50c66a None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:41.880768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd86fb9-cbc1-4c76-8d1a-3ea92c50c66a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:41.881071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd86fb9-cbc1-4c76-8d1a-3ea92c50c66a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:41.882441 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd86fb9-cbc1-4c76-8d1a-3ea92c50c66a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:41.890704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391298c-0360-4a8b-b2f5-aec4e74adcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:41.891337 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391298c-0360-4a8b-b2f5-aec4e74adcc8 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:41.891619 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391298c-0360-4a8b-b2f5-aec4e74adcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:41.891895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391298c-0360-4a8b-b2f5-aec4e74adcc8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:41.895243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 252/499] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:41.896497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48bda6-837f-4614-826b-3d29386686b8 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:41.896807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48bda6-837f-4614-826b-3d29386686b8 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:41.897078 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48bda6-837f-4614-826b-3d29386686b8 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:41.898468 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 253/500] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:41.899004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48bda6-837f-4614-826b-3d29386686b8 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:41.908193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391298c-0360-4a8b-b2f5-aec4e74adcc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:41.913515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-c391298c-0360-4a8b-b2f5-aec4e74adcc8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:41.913515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:41.913515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:41.913515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:41.913515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:41.913515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:41.913515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:41.913515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:41.913515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:41.913515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:41.913515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:41.913515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:41.913515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:41.913515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:41.913515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:41.913515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:41.914495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:41.914495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:41.914495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:41.914495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:41.914495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:41.914495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:41.914495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:41.914495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:41.914495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:41.914495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:41.914495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:41.914495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:41.914495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:41.914495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:41.914495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:41.914495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:41.915263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:41.915263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:41.915263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:41.915263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:41.915263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:41.915263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:41.915263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:41.915263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:41.915263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:41.915263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:41.915263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:41.915263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:41.915263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 248/501] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:41.915903 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391298c-0360-4a8b-b2f5-aec4e74adcc8 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:41.932011 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:42.006978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48bda6-837f-4614-826b-3d29386686b8 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:42.018078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391298c-0360-4a8b-b2f5-aec4e74adcc8 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:42.038203 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:42.048539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a46b3f-0481-4f70-8238-887be6894dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:42.049619 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a46b3f-0481-4f70-8238-887be6894dc4 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:42.050114 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a46b3f-0481-4f70-8238-887be6894dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:42.050573 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a46b3f-0481-4f70-8238-887be6894dc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:42.052154 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ace1a-3cb8-425c-addb-51c05c13594d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:42.052801 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ace1a-3cb8-425c-addb-51c05c13594d None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:42.053086 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ace1a-3cb8-425c-addb-51c05c13594d None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:42.053359 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ace1a-3cb8-425c-addb-51c05c13594d None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:42.054237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ace1a-3cb8-425c-addb-51c05c13594d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:42.054772 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ace1a-3cb8-425c-addb-51c05c13594d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:42.057435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ace1a-3cb8-425c-addb-51c05c13594d None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:42.063544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a46b3f-0481-4f70-8238-887be6894dc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:42.065961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a46b3f-0481-4f70-8238-887be6894dc4 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:42.088897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-69a46b3f-0481-4f70-8238-887be6894dc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:42.088897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:42.088897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:42.088897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:42.088897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:42.088897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:42.088897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:42.088897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:42.088897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:42.088897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:42.088897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:42.088897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:42.088897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:42.088897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:42.088897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:42.089861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:42.089861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:42.089861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:42.089861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:42.089861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:42.089861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:42.089861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:42.089861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:42.089861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:42.089861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:42.089861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:42.089861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:42.089861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:42.089861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:42.089861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:42.089861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:42.089861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:42.090693 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:42.090693 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:42.090693 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:42.090693 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:42.090693 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 254/502] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:42.090962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a46b3f-0481-4f70-8238-887be6894dc4 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:42.100653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 249/503] 10.222.0.45 () {56 vars in 1133 bytes} [Thu Aug 4 02:04:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:42.101898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cbcc1-7507-496d-bd9a-d97e21f86d36 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:42.102184 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cbcc1-7507-496d-bd9a-d97e21f86d36 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:42.102458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cbcc1-7507-496d-bd9a-d97e21f86d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:42.120128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-bc6cbcc1-7507-496d-bd9a-d97e21f86d36 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:42.121356 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-bc6cbcc1-7507-496d-bd9a-d97e21f86d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:42.178848 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:42.188588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 250/504] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:42.189536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cbcc1-7507-496d-bd9a-d97e21f86d36 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:42.199592 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a46b3f-0481-4f70-8238-887be6894dc4 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:42.234046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737df557-06a8-442d-96a7-1da74efdafda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:42.234680 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737df557-06a8-442d-96a7-1da74efdafda None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:42.234961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737df557-06a8-442d-96a7-1da74efdafda None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:42.235263 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737df557-06a8-442d-96a7-1da74efdafda None admin] PATH_INFO: `/v3/users/glance` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:42.243444 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737df557-06a8-442d-96a7-1da74efdafda None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:42.244919 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737df557-06a8-442d-96a7-1da74efdafda None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:42.257754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-737df557-06a8-442d-96a7-1da74efdafda None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 02:04:42.257754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:42.257754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:42.257754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:42.257754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:42.257754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:42.257754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:42.257754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:42.257754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:42.257754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:42.257754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:42.257754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:42.257754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:04:42.257754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:04:42.257754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:04:42.257754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:42.258870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:42.259995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:42.259995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:42.259995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:04:42.259995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:04:42.259995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:04:42.259995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:04:42.259995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:04:42.259995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:04:42.259995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 02:04:42.259995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:42.259995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 255/505] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:42 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:42.259995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737df557-06a8-442d-96a7-1da74efdafda None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:42.301940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cbcc1-7507-496d-bd9a-d97e21f86d36 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:42.311008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:42.335861 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f51944-86af-409d-9933-e03d373b6035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:42.336463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f51944-86af-409d-9933-e03d373b6035 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:42.336772 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f51944-86af-409d-9933-e03d373b6035 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:42.337051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f51944-86af-409d-9933-e03d373b6035 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:42.337763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f51944-86af-409d-9933-e03d373b6035 None admin] RBAC: Adding query filter params (name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:42.338067 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f51944-86af-409d-9933-e03d373b6035 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:42.339453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f51944-86af-409d-9933-e03d373b6035 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:42.348790 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 251/506] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:04:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:42.349589 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f51944-86af-409d-9933-e03d373b6035 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:42.365375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737df557-06a8-442d-96a7-1da74efdafda None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:42.399463 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e01fb-1af0-41f4-97f3-df048e89a9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:42.400068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e01fb-1af0-41f4-97f3-df048e89a9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:42.400349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e01fb-1af0-41f4-97f3-df048e89a9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:42.400651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e01fb-1af0-41f4-97f3-df048e89a9e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:42.401341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e01fb-1af0-41f4-97f3-df048e89a9e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:42.401646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e01fb-1af0-41f4-97f3-df048e89a9e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:42.403032 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e01fb-1af0-41f4-97f3-df048e89a9e4 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:42.412273 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:42.420973 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 256/507] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:42.421771 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e01fb-1af0-41f4-97f3-df048e89a9e4 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:42.455659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f51944-86af-409d-9933-e03d373b6035 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:42.489380 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031783bc-63fd-4a32-927b-98a158eb4d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:42.489994 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031783bc-63fd-4a32-927b-98a158eb4d96 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:42.490273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031783bc-63fd-4a32-927b-98a158eb4d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:42.490543 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031783bc-63fd-4a32-927b-98a158eb4d96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:42.525935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e01fb-1af0-41f4-97f3-df048e89a9e4 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:42.529744 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:42.558816 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031783bc-63fd-4a32-927b-98a158eb4d96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:42.559968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5341abb-4227-4c0a-9ab8-cc86bf1145e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:42.560273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031783bc-63fd-4a32-927b-98a158eb4d96 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:42.560601 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5341abb-4227-4c0a-9ab8-cc86bf1145e0 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:42.560910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5341abb-4227-4c0a-9ab8-cc86bf1145e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:42.561181 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5341abb-4227-4c0a-9ab8-cc86bf1145e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:42.571949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5341abb-4227-4c0a-9ab8-cc86bf1145e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:42.573431 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5341abb-4227-4c0a-9ab8-cc86bf1145e0 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:42.589285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-a5341abb-4227-4c0a-9ab8-cc86bf1145e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:42.589285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:42.589285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:42.589285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:42.589285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:42.589285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:42.589285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:42.589285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:42.589285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:42.589285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:42.589285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:42.589285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:42.589285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:42.589285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:42.589285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:42.590254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:42.590254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:42.590254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:42.590254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:42.590254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:42.590254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:42.590254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:42.590254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:42.590254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:42.590254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:42.590254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:42.590254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:42.590254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:42.590254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:42.590254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:42.590254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:42.590254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:42.591097 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:42.591097 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:42.591097 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:42.591097 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:42.591097 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 257/508] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:42.591354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5341abb-4227-4c0a-9ab8-cc86bf1145e0 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:42.642279 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:42.664139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 252/509] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:42.665018 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031783bc-63fd-4a32-927b-98a158eb4d96 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:42.693607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5341abb-4227-4c0a-9ab8-cc86bf1145e0 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:42.727996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa751ef-58d7-4626-bd12-17dae53d1b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:42.728533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa751ef-58d7-4626-bd12-17dae53d1b69 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:42.728846 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa751ef-58d7-4626-bd12-17dae53d1b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:42.729121 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa751ef-58d7-4626-bd12-17dae53d1b69 None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:42.729918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa751ef-58d7-4626-bd12-17dae53d1b69 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:42.730221 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa751ef-58d7-4626-bd12-17dae53d1b69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:42.731672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa751ef-58d7-4626-bd12-17dae53d1b69 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:42.732108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:42.762807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031783bc-63fd-4a32-927b-98a158eb4d96 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:42.768148 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 258/510] 10.222.0.45 () {56 vars in 1137 bytes} [Thu Aug 4 02:04:42 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:42.769106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa751ef-58d7-4626-bd12-17dae53d1b69 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:42.796180 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56938324-4b7c-4989-b52f-8738d40a451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:42.796836 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56938324-4b7c-4989-b52f-8738d40a451e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:42.797123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56938324-4b7c-4989-b52f-8738d40a451e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:42.797394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56938324-4b7c-4989-b52f-8738d40a451e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:42.807822 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56938324-4b7c-4989-b52f-8738d40a451e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:42.809308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56938324-4b7c-4989-b52f-8738d40a451e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:42.823883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-56938324-4b7c-4989-b52f-8738d40a451e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:42.823883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:42.823883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:42.823883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:42.823883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:42.823883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:42.823883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:42.823883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:42.823883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:42.823883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:42.823883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:42.823883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:42.823883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:42.823883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:42.823883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:42.824786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:42.824786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:42.824786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:42.824786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:42.824786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:42.824786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:42.824786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:42.824786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:42.824786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:42.824786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:42.824786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:42.824786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:42.824786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:42.824786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:42.824786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:42.824786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:42.824786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:42.825645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:42.825645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:42.825645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:42.825645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:42.825645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 253/511] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:42.826798 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56938324-4b7c-4989-b52f-8738d40a451e None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:42.868183 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa751ef-58d7-4626-bd12-17dae53d1b69 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:42.878531 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:42.902952 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2060a-aae7-4c22-b9b4-de240c034f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:42.903590 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2060a-aae7-4c22-b9b4-de240c034f1b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:42.903862 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2060a-aae7-4c22-b9b4-de240c034f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:42.904131 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2060a-aae7-4c22-b9b4-de240c034f1b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:42.911820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc2060a-aae7-4c22-b9b4-de240c034f1b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:42.913306 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc2060a-aae7-4c22-b9b4-de240c034f1b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:42.925477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-7fc2060a-aae7-4c22-b9b4-de240c034f1b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 02:04:42.925477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:42.925477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:42.925477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:42.925477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:42.925477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:42.925477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:42.925477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:42.925477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:42.925477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:42.925477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:42.925477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:42.925477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:04:42.925477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:04:42.925477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:04:42.925477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:42.926401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:42.927280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:42.927280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:42.927280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:04:42.927280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:04:42.927280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:04:42.927280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:04:42.927280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:04:42.927280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:04:42.927280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 02:04:42.927280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:42.927280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 259/512] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:42.927973 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2060a-aae7-4c22-b9b4-de240c034f1b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:42.933774 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56938324-4b7c-4989-b52f-8738d40a451e None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:42.957147 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:42.969460 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02d1e4-2021-45af-ab08-a6e799cde3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:42.970086 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02d1e4-2021-45af-ab08-a6e799cde3ca None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:42.970358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02d1e4-2021-45af-ab08-a6e799cde3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:42.970636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02d1e4-2021-45af-ab08-a6e799cde3ca None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:42.971332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02d1e4-2021-45af-ab08-a6e799cde3ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:42.971684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02d1e4-2021-45af-ab08-a6e799cde3ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:42.973590 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c02d1e4-2021-45af-ab08-a6e799cde3ca None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:42.985448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 254/513] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:42.986236 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02d1e4-2021-45af-ab08-a6e799cde3ca None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:43.031225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2060a-aae7-4c22-b9b4-de240c034f1b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:43.065291 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14fa21-d6df-4585-9184-a979b0faf62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:43.065972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14fa21-d6df-4585-9184-a979b0faf62d None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:43.066251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14fa21-d6df-4585-9184-a979b0faf62d None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:43.066523 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a14fa21-d6df-4585-9184-a979b0faf62d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:43.077046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14fa21-d6df-4585-9184-a979b0faf62d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:43.078499 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a14fa21-d6df-4585-9184-a979b0faf62d None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:43.085070 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02d1e4-2021-45af-ab08-a6e799cde3ca None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:43.093370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-1a14fa21-d6df-4585-9184-a979b0faf62d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:43.093370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:43.093370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:43.093370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:43.093370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:43.093370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:43.093370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:43.093370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:43.093370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:43.093370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:43.093370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:43.093370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:43.093370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:43.093370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:43.093370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:43.094203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:43.094203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:43.094203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:43.094203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:43.094203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:43.094203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:43.094203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:43.094203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:43.094203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:43.094203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:43.094203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:43.094203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:43.094203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:43.094203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:43.094203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:43.094203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:43.094203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:43.095044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:43.095044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:43.095044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:43.095044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:43.095044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 260/514] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:43.095306 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14fa21-d6df-4585-9184-a979b0faf62d None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:43.119578 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:43.121319 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c6931-ae3d-4864-9fd3-20087ff34cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:43.121966 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c6931-ae3d-4864-9fd3-20087ff34cfb None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:43.122222 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c6931-ae3d-4864-9fd3-20087ff34cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:43.122499 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c6931-ae3d-4864-9fd3-20087ff34cfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:43.170058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:43.189178 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c6931-ae3d-4864-9fd3-20087ff34cfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:43.191176 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084c6931-ae3d-4864-9fd3-20087ff34cfb None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:43.197043 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a14fa21-d6df-4585-9184-a979b0faf62d None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:43.229451 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b671b-7256-477e-b9db-db9f8908d70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:43.230062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b671b-7256-477e-b9db-db9f8908d70d None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:43.230343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b671b-7256-477e-b9db-db9f8908d70d None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:43.230616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721b671b-7256-477e-b9db-db9f8908d70d None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:43.231304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b671b-7256-477e-b9db-db9f8908d70d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:43.231610 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b671b-7256-477e-b9db-db9f8908d70d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:43.233004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721b671b-7256-477e-b9db-db9f8908d70d None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:43.245038 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 261/515] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:43.245771 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b671b-7256-477e-b9db-db9f8908d70d None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:43.292084 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 255/516] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:43.292948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c6931-ae3d-4864-9fd3-20087ff34cfb None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:43.338139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:43.357578 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721b671b-7256-477e-b9db-db9f8908d70d None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:43.373898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:43.392113 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed8dd5-f6ac-4c1c-b96c-24623872db60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:43.392763 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed8dd5-f6ac-4c1c-b96c-24623872db60 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:43.393047 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed8dd5-f6ac-4c1c-b96c-24623872db60 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:43.393282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c6931-ae3d-4864-9fd3-20087ff34cfb None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:43.393387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed8dd5-f6ac-4c1c-b96c-24623872db60 None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:43.394162 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed8dd5-f6ac-4c1c-b96c-24623872db60 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:43.394471 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed8dd5-f6ac-4c1c-b96c-24623872db60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:43.395909 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fed8dd5-f6ac-4c1c-b96c-24623872db60 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:43.414423 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 262/517] 10.222.0.45 () {56 vars in 1137 bytes} [Thu Aug 4 02:04:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:43.415247 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed8dd5-f6ac-4c1c-b96c-24623872db60 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:43.430862 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8e9fe-c781-4272-be68-7583450d6fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:43.431475 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8e9fe-c781-4272-be68-7583450d6fb3 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:43.431758 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8e9fe-c781-4272-be68-7583450d6fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:43.432034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad8e9fe-c781-4272-be68-7583450d6fb3 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:43.432734 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8e9fe-c781-4272-be68-7583450d6fb3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:43.434166 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad8e9fe-c781-4272-be68-7583450d6fb3 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:43.456129 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 256/518] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:04:43 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:04:43.456983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8e9fe-c781-4272-be68-7583450d6fb3 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:43.536386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed8dd5-f6ac-4c1c-b96c-24623872db60 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:43.571797 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad8e9fe-c781-4272-be68-7583450d6fb3 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:43.574836 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948614e-053e-444d-a0bf-ad7f9607f8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:43.575470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948614e-053e-444d-a0bf-ad7f9607f8fb None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:43.575746 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948614e-053e-444d-a0bf-ad7f9607f8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:43.576016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b948614e-053e-444d-a0bf-ad7f9607f8fb None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:43.576767 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948614e-053e-444d-a0bf-ad7f9607f8fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:43.577075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948614e-053e-444d-a0bf-ad7f9607f8fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:43.578578 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b948614e-053e-444d-a0bf-ad7f9607f8fb None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:43.587912 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:43.596524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 263/519] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:43.598035 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948614e-053e-444d-a0bf-ad7f9607f8fb None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:43.612102 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f332892-bcf8-4189-bec3-b4588b783ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:43.613345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f332892-bcf8-4189-bec3-b4588b783ffc None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:43.613627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f332892-bcf8-4189-bec3-b4588b783ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:43.613902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f332892-bcf8-4189-bec3-b4588b783ffc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:43.624548 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f332892-bcf8-4189-bec3-b4588b783ffc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:43.625806 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f332892-bcf8-4189-bec3-b4588b783ffc None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:43.628252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:43.640119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-4f332892-bcf8-4189-bec3-b4588b783ffc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 02:04:43.640119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:43.640119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:43.640119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:43.640119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:43.640119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:43.640119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:43.640119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:43.640119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:43.640119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:43.640119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:43.640119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:43.640119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:04:43.640119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:04:43.640119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:04:43.640119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:43.641092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:43.641984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:43.641984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:43.641984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:04:43.641984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:04:43.641984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:04:43.641984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:04:43.641984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:04:43.641984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:04:43.641984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 02:04:43.641984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:43.641984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 257/520] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:43.643017 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f332892-bcf8-4189-bec3-b4588b783ffc None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:43.716507 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b948614e-053e-444d-a0bf-ad7f9607f8fb None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:43.752316 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057757e-ae92-4e10-9572-90264c7f2003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:43.753050 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057757e-ae92-4e10-9572-90264c7f2003 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:43.753333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057757e-ae92-4e10-9572-90264c7f2003 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:43.753687 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f057757e-ae92-4e10-9572-90264c7f2003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:43.760532 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f332892-bcf8-4189-bec3-b4588b783ffc None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:43.765561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057757e-ae92-4e10-9572-90264c7f2003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:43.767043 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f057757e-ae92-4e10-9572-90264c7f2003 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:43.783562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-f057757e-ae92-4e10-9572-90264c7f2003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:43.783562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:43.783562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:43.783562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:43.783562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:43.783562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:43.783562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:43.783562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:43.783562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:43.783562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:43.783562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:43.783562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:43.783562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:43.783562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:43.783562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:43.785123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:43.785123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:43.785123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:43.785123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:43.785123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:43.785123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:43.785123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:43.785123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:43.785123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:43.785123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:43.785123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:43.785123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:43.785123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:43.785123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:43.785123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:43.785123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:43.785123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:43.785969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:43.785969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:43.785969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:43.785969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:43.785969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 264/521] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:43.786234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057757e-ae92-4e10-9572-90264c7f2003 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:43.801394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94ced8-a7ce-4ddd-8186-3ce80b38d152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:43.801939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94ced8-a7ce-4ddd-8186-3ce80b38d152 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:43.802225 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94ced8-a7ce-4ddd-8186-3ce80b38d152 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:43.802511 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea94ced8-a7ce-4ddd-8186-3ce80b38d152 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:43.842812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:43.875178 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:43.877647 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94ced8-a7ce-4ddd-8186-3ce80b38d152 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:43.879520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea94ced8-a7ce-4ddd-8186-3ce80b38d152 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:43.898302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f057757e-ae92-4e10-9572-90264c7f2003 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:43.935915 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40d37f-9446-4cbf-ba2b-0395728a4fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:43.936939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40d37f-9446-4cbf-ba2b-0395728a4fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:43.937235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40d37f-9446-4cbf-ba2b-0395728a4fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:43.937571 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40d37f-9446-4cbf-ba2b-0395728a4fb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:43.946854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40d37f-9446-4cbf-ba2b-0395728a4fb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:43.948428 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40d37f-9446-4cbf-ba2b-0395728a4fb1 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:43.963839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-ee40d37f-9446-4cbf-ba2b-0395728a4fb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:43.963839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:43.963839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:43.963839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:43.963839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:43.963839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:43.963839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:43.963839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:43.963839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:43.963839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:43.963839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:43.963839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:43.963839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:04:43.963839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:04:43.963839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:04:43.964785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:04:43.964785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:43.964785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:43.964785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:43.964785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:43.964785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:43.964785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:43.964785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:43.964785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:43.964785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:43.964785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:43.964785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:43.964785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:43.964785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:43.964785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:43.964785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:43.964785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:43.965628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:04:43.965628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:04:43.965628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:04:43.965628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:04:43.965628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:04:43.965628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:04:43.965628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:43.965628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:43.965628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 265/522] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:43.965628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40d37f-9446-4cbf-ba2b-0395728a4fb1 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:43.994806 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 258/523] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:43.995651 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94ced8-a7ce-4ddd-8186-3ce80b38d152 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:44.077893 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40d37f-9446-4cbf-ba2b-0395728a4fb1 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:44.096661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:44.108240 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:44.112702 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea94ced8-a7ce-4ddd-8186-3ce80b38d152 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:44.122874 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971173b-ea33-47d5-a8c4-b93d1868041b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:44.123837 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971173b-ea33-47d5-a8c4-b93d1868041b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:44.124114 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971173b-ea33-47d5-a8c4-b93d1868041b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:44.124387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0971173b-ea33-47d5-a8c4-b93d1868041b None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:44.125516 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971173b-ea33-47d5-a8c4-b93d1868041b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:44.125821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971173b-ea33-47d5-a8c4-b93d1868041b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:44.127310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0971173b-ea33-47d5-a8c4-b93d1868041b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:44.146195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 266/524] 10.222.0.45 () {56 vars in 1139 bytes} [Thu Aug 4 02:04:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:44.147102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971173b-ea33-47d5-a8c4-b93d1868041b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:44.155128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdee4c1-5cc8-438e-9256-3b7f01cfbe54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:44.156029 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdee4c1-5cc8-438e-9256-3b7f01cfbe54 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:44.156601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdee4c1-5cc8-438e-9256-3b7f01cfbe54 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:44.157065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdee4c1-5cc8-438e-9256-3b7f01cfbe54 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:44.157926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdee4c1-5cc8-438e-9256-3b7f01cfbe54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:44.158377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdee4c1-5cc8-438e-9256-3b7f01cfbe54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:44.160125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdee4c1-5cc8-438e-9256-3b7f01cfbe54 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:44.173209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 259/525] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:44.174023 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdee4c1-5cc8-438e-9256-3b7f01cfbe54 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:44.259303 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0971173b-ea33-47d5-a8c4-b93d1868041b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:44.284532 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdee4c1-5cc8-438e-9256-3b7f01cfbe54 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:44.296667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfd5b4-e875-4b35-8591-767a6330f609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:44.297310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfd5b4-e875-4b35-8591-767a6330f609 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:44.297594 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfd5b4-e875-4b35-8591-767a6330f609 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:44.297867 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfd5b4-e875-4b35-8591-767a6330f609 None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:44.298616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfd5b4-e875-4b35-8591-767a6330f609 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:44.298921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfd5b4-e875-4b35-8591-767a6330f609 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:44.300508 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfd5b4-e875-4b35-8591-767a6330f609 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:44.324065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 267/526] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:04:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:44.324065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfd5b4-e875-4b35-8591-767a6330f609 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:44.325277 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cac195-a1ca-4b4c-9478-06434ee5bc16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:44.325923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cac195-a1ca-4b4c-9478-06434ee5bc16 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:44.326206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cac195-a1ca-4b4c-9478-06434ee5bc16 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:44.326481 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cac195-a1ca-4b4c-9478-06434ee5bc16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:44.338166 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cac195-a1ca-4b4c-9478-06434ee5bc16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:44.339551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cac195-a1ca-4b4c-9478-06434ee5bc16 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:44.354907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-43cac195-a1ca-4b4c-9478-06434ee5bc16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:44.354907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:44.354907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:44.354907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:44.354907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:44.354907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:44.354907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:44.354907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:44.354907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:44.354907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:44.354907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:44.354907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:44.354907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:44.354907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:44.354907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:44.355743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:44.355743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:44.355743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:44.355743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:44.355743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:44.355743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:44.355743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:44.355743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:44.355743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:44.355743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:44.355743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:44.355743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:44.355743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:44.355743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:44.355743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:44.355743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:44.355743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:44.356574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:44.356574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:44.356574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:44.356574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:44.356574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 260/527] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:44.356883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cac195-a1ca-4b4c-9478-06434ee5bc16 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:44.358149 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:44.378390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:44.443038 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfd5b4-e875-4b35-8591-767a6330f609 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:44.477610 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cac195-a1ca-4b4c-9478-06434ee5bc16 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:44.485112 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede43aab-3ad4-451e-bb3b-f9cb8c364687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:44.485734 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede43aab-3ad4-451e-bb3b-f9cb8c364687 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:44.486438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede43aab-3ad4-451e-bb3b-f9cb8c364687 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:44.486732 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede43aab-3ad4-451e-bb3b-f9cb8c364687 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:44.515551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70955a-9ab2-4bf7-8650-c4b5c7bcfb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:44.516214 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70955a-9ab2-4bf7-8650-c4b5c7bcfb95 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:44.516497 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70955a-9ab2-4bf7-8650-c4b5c7bcfb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:44.517116 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70955a-9ab2-4bf7-8650-c4b5c7bcfb95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:44.524525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c70955a-9ab2-4bf7-8650-c4b5c7bcfb95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:44.525990 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c70955a-9ab2-4bf7-8650-c4b5c7bcfb95 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:44.539060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-1c70955a-9ab2-4bf7-8650-c4b5c7bcfb95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:44.539060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:44.539060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:44.539060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:44.539060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:44.539060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:44.539060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:44.539060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:44.539060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:44.539060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:44.539060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:44.539060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:44.539060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:04:44.539060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:04:44.539060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:04:44.540019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:04:44.540019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:44.540019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:44.540019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:44.540019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:44.540019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:44.540019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:44.540019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:44.540019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:44.540019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:44.540019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:44.540019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:44.540019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:44.540019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:44.540019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:44.540019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:44.540019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:44.545711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:04:44.545711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:04:44.545711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:04:44.545711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:04:44.545711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:04:44.545711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:04:44.545711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:44.545711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:44.545711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 261/528] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:44.545711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70955a-9ab2-4bf7-8650-c4b5c7bcfb95 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:44.567610 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede43aab-3ad4-451e-bb3b-f9cb8c364687 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:44.569353 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede43aab-3ad4-451e-bb3b-f9cb8c364687 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:44.602495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:44.618233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:44.657901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70955a-9ab2-4bf7-8650-c4b5c7bcfb95 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:44.692043 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 268/529] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:44.692043 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede43aab-3ad4-451e-bb3b-f9cb8c364687 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:44.703677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75daeb97-8bde-48bb-9dbc-45aa51d75a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:44.704303 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daeb97-8bde-48bb-9dbc-45aa51d75a34 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:44.704584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daeb97-8bde-48bb-9dbc-45aa51d75a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:44.704886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daeb97-8bde-48bb-9dbc-45aa51d75a34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:44.705683 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75daeb97-8bde-48bb-9dbc-45aa51d75a34 None admin] RBAC: Adding query filter params (role.id=977824a8c6a24c00af2f6dabd0e2a6f0, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=c571406d2a3f40dab035a918399d6f6a) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:44.706007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75daeb97-8bde-48bb-9dbc-45aa51d75a34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:44.707451 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75daeb97-8bde-48bb-9dbc-45aa51d75a34 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:44.713500 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 262/530] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:44 2022] GET /identity/v3/role_assignments?user.id=c571406d2a3f40dab035a918399d6f6a&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=977824a8c6a24c00af2f6dabd0e2a6f0 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:44.717585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75daeb97-8bde-48bb-9dbc-45aa51d75a34 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:44.820306 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede43aab-3ad4-451e-bb3b-f9cb8c364687 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:44.854616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:44.861836 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f51e5-57c6-4bfe-bc4b-f6e1adef55c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:44.863127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f51e5-57c6-4bfe-bc4b-f6e1adef55c6 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:44.863468 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f51e5-57c6-4bfe-bc4b-f6e1adef55c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:44.863807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f51e5-57c6-4bfe-bc4b-f6e1adef55c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:44.864581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f51e5-57c6-4bfe-bc4b-f6e1adef55c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:44.865012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f51e5-57c6-4bfe-bc4b-f6e1adef55c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:44.865812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75daeb97-8bde-48bb-9dbc-45aa51d75a34 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:44.866614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f51e5-57c6-4bfe-bc4b-f6e1adef55c6 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:44.880930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 269/531] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:44.881798 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f51e5-57c6-4bfe-bc4b-f6e1adef55c6 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:44.914142 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881d623-5dbb-4e0e-9e23-45ecbc1c5712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:44.914788 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881d623-5dbb-4e0e-9e23-45ecbc1c5712 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:44.915076 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881d623-5dbb-4e0e-9e23-45ecbc1c5712 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:44.915349 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881d623-5dbb-4e0e-9e23-45ecbc1c5712 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:44.916158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881d623-5dbb-4e0e-9e23-45ecbc1c5712 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:44.916407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881d623-5dbb-4e0e-9e23-45ecbc1c5712 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:44.917949 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881d623-5dbb-4e0e-9e23-45ecbc1c5712 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:44.938928 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 263/532] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:04:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:44.939958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881d623-5dbb-4e0e-9e23-45ecbc1c5712 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:44.968998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:45.018740 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f51e5-57c6-4bfe-bc4b-f6e1adef55c6 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:45.045832 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881d623-5dbb-4e0e-9e23-45ecbc1c5712 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:45.062310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37017eb9-3217-4a89-8b9d-dd3da523c5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:45.063116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37017eb9-3217-4a89-8b9d-dd3da523c5bb None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:45.063496 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37017eb9-3217-4a89-8b9d-dd3da523c5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:45.063866 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37017eb9-3217-4a89-8b9d-dd3da523c5bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:45.075254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37017eb9-3217-4a89-8b9d-dd3da523c5bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:45.077125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37017eb9-3217-4a89-8b9d-dd3da523c5bb None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:45.089702 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9627d3d-d026-449e-9933-3fe90bdbe834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:45.090588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9627d3d-d026-449e-9933-3fe90bdbe834 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:45.091014 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9627d3d-d026-449e-9933-3fe90bdbe834 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:45.091446 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9627d3d-d026-449e-9933-3fe90bdbe834 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:45.093234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-37017eb9-3217-4a89-8b9d-dd3da523c5bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:45.093234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:45.093234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:45.093234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:45.093234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:45.093234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:45.093234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:45.093234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:45.093234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:45.093234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:45.093234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:45.093234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:45.093234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:45.093234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:45.093234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:45.094092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:45.094092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:45.094092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:45.094092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:45.094092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:45.094092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:45.094092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:45.094092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:45.094092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:45.094092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:45.094092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:45.094092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:45.094092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:45.094092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:45.094092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:45.094092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:45.094092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:45.094933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:45.094933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:45.094933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:45.094933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:45.099877 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 270/533] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:45.100924 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37017eb9-3217-4a89-8b9d-dd3da523c5bb None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:45.101987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9627d3d-d026-449e-9933-3fe90bdbe834 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:45.103623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9627d3d-d026-449e-9933-3fe90bdbe834 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:45.119258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-f9627d3d-d026-449e-9933-3fe90bdbe834 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:45.119258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:45.119258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:45.119258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:45.119258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:45.119258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:45.119258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:45.119258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:45.119258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:45.119258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:45.119258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:45.119258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:45.119258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:04:45.119258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:04:45.119258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:04:45.120159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:04:45.120159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:45.120159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:45.120159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:45.120159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:45.120159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:45.120159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:45.120159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:45.120159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:45.120159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:45.120159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:45.120159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:45.120159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:45.120159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:45.120159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:45.120159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:45.120159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:45.121051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:04:45.121051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:04:45.121051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:04:45.121051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:04:45.121051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:04:45.121051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:04:45.121051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:45.121051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:45.122341 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 264/534] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:45.123384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9627d3d-d026-449e-9933-3fe90bdbe834 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:45.162952 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:45.198130 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:45.228427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37017eb9-3217-4a89-8b9d-dd3da523c5bb None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:45.256475 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9627d3d-d026-449e-9933-3fe90bdbe834 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:45.266214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0dc247-32de-4dbc-bf36-6c184e24ce03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:45.267263 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dc247-32de-4dbc-bf36-6c184e24ce03 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:45.267747 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dc247-32de-4dbc-bf36-6c184e24ce03 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:45.268239 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dc247-32de-4dbc-bf36-6c184e24ce03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:45.269755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dc247-32de-4dbc-bf36-6c184e24ce03 None admin] RBAC: Adding query filter params (role.id=977824a8c6a24c00af2f6dabd0e2a6f0, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=14ff31d52ecf4c7d93a65bbfca7285c0) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:45.270298 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dc247-32de-4dbc-bf36-6c184e24ce03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:45.272049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dc247-32de-4dbc-bf36-6c184e24ce03 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:45.278627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 271/535] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:45 2022] GET /identity/v3/role_assignments?user.id=14ff31d52ecf4c7d93a65bbfca7285c0&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=977824a8c6a24c00af2f6dabd0e2a6f0 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:45.279875 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0dc247-32de-4dbc-bf36-6c184e24ce03 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:45.300827 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ecece7-fc64-4c70-9c3b-29cb2c2eaffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:45.301920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecece7-fc64-4c70-9c3b-29cb2c2eaffb None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:45.302404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecece7-fc64-4c70-9c3b-29cb2c2eaffb None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:45.302894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ecece7-fc64-4c70-9c3b-29cb2c2eaffb None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:45.303916 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ecece7-fc64-4c70-9c3b-29cb2c2eaffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:45.304512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ecece7-fc64-4c70-9c3b-29cb2c2eaffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:45.313145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ecece7-fc64-4c70-9c3b-29cb2c2eaffb None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:45.333506 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 265/536] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:45.339019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ecece7-fc64-4c70-9c3b-29cb2c2eaffb None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:45.503680 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ecece7-fc64-4c70-9c3b-29cb2c2eaffb None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:45.517189 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0dc247-32de-4dbc-bf36-6c184e24ce03 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:45.528828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:45.530601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:45.570061 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10799a4c-663a-4470-b972-d81a51cead24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:45.570703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10799a4c-663a-4470-b972-d81a51cead24 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:45.570982 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10799a4c-663a-4470-b972-d81a51cead24 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:45.571258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10799a4c-663a-4470-b972-d81a51cead24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:45.580020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10799a4c-663a-4470-b972-d81a51cead24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:45.581943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10799a4c-663a-4470-b972-d81a51cead24 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:45.583839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a6891-169c-44ee-8fb8-810986ea655f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:45.584461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6891-169c-44ee-8fb8-810986ea655f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:45.584768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6891-169c-44ee-8fb8-810986ea655f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:45.585044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a6891-169c-44ee-8fb8-810986ea655f None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:45.585806 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a6891-169c-44ee-8fb8-810986ea655f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:45.586125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a6891-169c-44ee-8fb8-810986ea655f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:45.587520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5a6891-169c-44ee-8fb8-810986ea655f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:45.602531 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 272/537] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:04:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:45.607209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-10799a4c-663a-4470-b972-d81a51cead24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:45.607209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:45.607209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:45.607209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:45.607209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:45.607209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:45.607209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:45.607209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:45.607209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:45.607209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:45.607209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:45.607209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:45.607209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:04:45.607209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:04:45.607209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:04:45.608050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:04:45.608050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:45.608050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:45.608050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:45.608050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:45.608050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:45.608050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:45.608050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:45.608050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:45.608050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:45.608050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:45.608050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:45.608050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:45.608050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:45.608050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:45.608050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:45.608050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:45.609001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:04:45.609001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:04:45.609001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:04:45.609001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:04:45.609001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:04:45.609001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:04:45.609001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:45.609001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:45.609852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a6891-169c-44ee-8fb8-810986ea655f None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:45.615433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 266/538] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:04:45.618266 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10799a4c-663a-4470-b972-d81a51cead24 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:45.744333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10799a4c-663a-4470-b972-d81a51cead24 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:45.752848 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a6891-169c-44ee-8fb8-810986ea655f None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:45.789989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e96d9-73ad-4570-8b36-233d29ef2d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:45.790839 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e96d9-73ad-4570-8b36-233d29ef2d29 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:45.791122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e96d9-73ad-4570-8b36-233d29ef2d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:45.791388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e96d9-73ad-4570-8b36-233d29ef2d29 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:45.792986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e96d9-73ad-4570-8b36-233d29ef2d29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:45.793716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e96d9-73ad-4570-8b36-233d29ef2d29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:45.797136 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651e96d9-73ad-4570-8b36-233d29ef2d29 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:45.806093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e2feb-0708-48d8-b12f-a2690f7ecac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:45.806758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e2feb-0708-48d8-b12f-a2690f7ecac6 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:45.807041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e2feb-0708-48d8-b12f-a2690f7ecac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:45.807426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e2feb-0708-48d8-b12f-a2690f7ecac6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:45.808244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e2feb-0708-48d8-b12f-a2690f7ecac6 None admin] RBAC: Adding query filter params (role.id=977824a8c6a24c00af2f6dabd0e2a6f0, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=f4b8838129174fbb8cdd32bd5d4d6101) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:45.808555 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e2feb-0708-48d8-b12f-a2690f7ecac6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:45.810151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e2feb-0708-48d8-b12f-a2690f7ecac6 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:45.814702 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 267/539] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:04:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:45.816112 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71f7e1-e0d2-42e8-b0b9-63bb8ae829f8 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:45.816394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71f7e1-e0d2-42e8-b0b9-63bb8ae829f8 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:45.816697 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71f7e1-e0d2-42e8-b0b9-63bb8ae829f8 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:45.820756 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 268/540] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:45.822128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71f7e1-e0d2-42e8-b0b9-63bb8ae829f8 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:45.825100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 273/541] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:45 2022] GET /identity/v3/role_assignments?user.id=f4b8838129174fbb8cdd32bd5d4d6101&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=977824a8c6a24c00af2f6dabd0e2a6f0 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:45.826434 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77ba2-4175-48b9-8be9-ab29640354e5 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:45.826835 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77ba2-4175-48b9-8be9-ab29640354e5 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:45.827114 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c77ba2-4175-48b9-8be9-ab29640354e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:45.852427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-56c77ba2-4175-48b9-8be9-ab29640354e5 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:45.853805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-56c77ba2-4175-48b9-8be9-ab29640354e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:45.868077 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:45.888276 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:45.949834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 274/542] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:45.973986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71f7e1-e0d2-42e8-b0b9-63bb8ae829f8 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:46.009518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c77ba2-4175-48b9-8be9-ab29640354e5 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:46.039537 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db562f03-ebf9-427c-82ed-39de2c60f4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:46.040239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db562f03-ebf9-427c-82ed-39de2c60f4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:46.040435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db562f03-ebf9-427c-82ed-39de2c60f4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:46.040751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db562f03-ebf9-427c-82ed-39de2c60f4e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:46.041555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db562f03-ebf9-427c-82ed-39de2c60f4e4 None admin] RBAC: Adding query filter params (role.id=977824a8c6a24c00af2f6dabd0e2a6f0, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=6c2da427acb24c8cb6d667c6347858c7) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:46.041859 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db562f03-ebf9-427c-82ed-39de2c60f4e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:46.043299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db562f03-ebf9-427c-82ed-39de2c60f4e4 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:46.054962 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 269/543] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:45 2022] GET /identity/v3/role_assignments?user.id=6c2da427acb24c8cb6d667c6347858c7&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=977824a8c6a24c00af2f6dabd0e2a6f0 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:46.179002 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c77ba2-4175-48b9-8be9-ab29640354e5 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:46.239980 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474ad1b-f9d1-4f75-886f-a70441f9fa72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:46.240664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474ad1b-f9d1-4f75-886f-a70441f9fa72 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:46.240956 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474ad1b-f9d1-4f75-886f-a70441f9fa72 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:46.241229 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474ad1b-f9d1-4f75-886f-a70441f9fa72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:46.259706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474ad1b-f9d1-4f75-886f-a70441f9fa72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:46.265022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-1474ad1b-f9d1-4f75-886f-a70441f9fa72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:46.265022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:46.265022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:46.265022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:46.265022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:46.265022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:46.265022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:46.265022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:46.265022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:46.265022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:46.265022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:46.265022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:46.265022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:46.265022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:46.265022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:46.265022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:46.265895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:46.265895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:46.265895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:46.265895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:46.265895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:46.265895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:46.265895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:46.265895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:46.265895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:46.265895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:46.265895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:46.265895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:46.265895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:46.265895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:46.265895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:46.265895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:46.266679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:46.266679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:46.266679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:46.266679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:46.266679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:46.266679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:46.266679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:46.266679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:46.266679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:46.266679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:46.266679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:46.266679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:46.269608 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 275/544] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:04:46.276663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db562f03-ebf9-427c-82ed-39de2c60f4e4 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:46.396338 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:46.401134 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db562f03-ebf9-427c-82ed-39de2c60f4e4 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:46.447289 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ff5db-fd1d-40f5-80ad-0e3ff322d9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:46.448161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ff5db-fd1d-40f5-80ad-0e3ff322d9cd None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:46.448589 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ff5db-fd1d-40f5-80ad-0e3ff322d9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:46.449027 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ff5db-fd1d-40f5-80ad-0e3ff322d9cd None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:46.449929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ff5db-fd1d-40f5-80ad-0e3ff322d9cd None admin] RBAC: Adding query filter params (name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:46.450380 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ff5db-fd1d-40f5-80ad-0e3ff322d9cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:46.451898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ff5db-fd1d-40f5-80ad-0e3ff322d9cd None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:46.464255 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 270/545] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:04:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:46.469317 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474ad1b-f9d1-4f75-886f-a70441f9fa72 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:46.512824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:46.613046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474ad1b-f9d1-4f75-886f-a70441f9fa72 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:46.673545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99daa77-14a3-4846-846c-84f3de61b266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:46.674189 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99daa77-14a3-4846-846c-84f3de61b266 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:46.674470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99daa77-14a3-4846-846c-84f3de61b266 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:46.675288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99daa77-14a3-4846-846c-84f3de61b266 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:46.692492 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99daa77-14a3-4846-846c-84f3de61b266 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:46.693997 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99daa77-14a3-4846-846c-84f3de61b266 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:46.711881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-b99daa77-14a3-4846-846c-84f3de61b266 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:46.711881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:46.711881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:46.711881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:46.711881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:46.711881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:46.711881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:46.711881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:46.711881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:46.711881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:46.711881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:46.711881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:46.711881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:46.711881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:46.711881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:46.712768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:46.712768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:46.712768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:46.712768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:46.712768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:46.712768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:46.712768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:46.712768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:46.712768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:46.712768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:46.712768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:46.712768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:46.712768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:46.712768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:46.712768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:46.712768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:46.712768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:46.713599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:46.713599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:46.713599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:46.713599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:46.713599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 276/546] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:46.719921 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ff5db-fd1d-40f5-80ad-0e3ff322d9cd None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:46.881780 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ff5db-fd1d-40f5-80ad-0e3ff322d9cd None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:46.924131 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc95966-08bc-4c25-9771-f131c16dfacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:46.925224 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc95966-08bc-4c25-9771-f131c16dfacb None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:46.925509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc95966-08bc-4c25-9771-f131c16dfacb None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:46.925783 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc95966-08bc-4c25-9771-f131c16dfacb None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:46.926491 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc95966-08bc-4c25-9771-f131c16dfacb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:46.926794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc95966-08bc-4c25-9771-f131c16dfacb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:46.928169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc95966-08bc-4c25-9771-f131c16dfacb None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:46.945651 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 271/547] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:46.951487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99daa77-14a3-4846-846c-84f3de61b266 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:46.955289 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7465696-587a-41ce-8eca-55519b68a3c7 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:46.955578 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7465696-587a-41ce-8eca-55519b68a3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:46.955853 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7465696-587a-41ce-8eca-55519b68a3c7 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:46.957486 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 272/548] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:04:46.962474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253545-37cd-40d6-b46f-b0c58d0874a1 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:46.962750 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253545-37cd-40d6-b46f-b0c58d0874a1 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:46.963021 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55253545-37cd-40d6-b46f-b0c58d0874a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:46.983848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-55253545-37cd-40d6-b46f-b0c58d0874a1 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:46.985093 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-55253545-37cd-40d6-b46f-b0c58d0874a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:47.000492 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:47.041259 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:47.073625 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99daa77-14a3-4846-846c-84f3de61b266 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:47.106789 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 273/549] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:04:47.111524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca12a3c-074d-4f38-b97f-02ad06f0c6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:47.112777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca12a3c-074d-4f38-b97f-02ad06f0c6fa None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:47.113076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca12a3c-074d-4f38-b97f-02ad06f0c6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:47.113353 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca12a3c-074d-4f38-b97f-02ad06f0c6fa None admin] PATH_INFO: `/v3/users/swift` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:47.121334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca12a3c-074d-4f38-b97f-02ad06f0c6fa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:47.122798 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca12a3c-074d-4f38-b97f-02ad06f0c6fa None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:47.134397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-2ca12a3c-074d-4f38-b97f-02ad06f0c6fa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 02:04:47.134397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:47.134397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:47.134397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:47.134397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:47.134397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:47.134397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:47.134397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:47.134397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:47.134397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:47.134397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:47.134397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:47.134397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:04:47.134397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:04:47.134397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:04:47.134397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:47.135820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:47.136929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:47.136929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:47.136929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:04:47.136929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:04:47.136929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:04:47.136929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:04:47.136929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:04:47.136929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:04:47.136929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 02:04:47.136929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:47.136929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 277/550] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:04:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:47.149688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca12a3c-074d-4f38-b97f-02ad06f0c6fa None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:47.149952 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55253545-37cd-40d6-b46f-b0c58d0874a1 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:47.309772 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55253545-37cd-40d6-b46f-b0c58d0874a1 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:47.346305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca12a3c-074d-4f38-b97f-02ad06f0c6fa None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:47.377531 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:47.384831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13155767-c0ff-4122-8743-1fc517bde764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:47.385455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13155767-c0ff-4122-8743-1fc517bde764 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:47.385737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13155767-c0ff-4122-8743-1fc517bde764 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:47.386010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13155767-c0ff-4122-8743-1fc517bde764 None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:47.386814 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13155767-c0ff-4122-8743-1fc517bde764 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:47.392642 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13155767-c0ff-4122-8743-1fc517bde764 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:47.392642 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13155767-c0ff-4122-8743-1fc517bde764 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:47.401325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553216e6-e2db-4088-929a-bc5b180a256e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:47.402281 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553216e6-e2db-4088-929a-bc5b180a256e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:47.402755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553216e6-e2db-4088-929a-bc5b180a256e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:47.403221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553216e6-e2db-4088-929a-bc5b180a256e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:47.407487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 278/551] 10.222.0.45 () {56 vars in 1135 bytes} [Thu Aug 4 02:04:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:47.413465 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13155767-c0ff-4122-8743-1fc517bde764 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:47.429536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553216e6-e2db-4088-929a-bc5b180a256e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:47.434834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-553216e6-e2db-4088-929a-bc5b180a256e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:47.434834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:47.434834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:47.434834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:47.434834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:47.434834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:47.434834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:47.434834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:47.434834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:47.434834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:47.434834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:47.434834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:47.434834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:47.434834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:47.434834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:47.434834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:47.435746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:47.435746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:47.435746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:47.435746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:47.435746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:47.435746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:47.435746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:47.435746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:47.435746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:47.435746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:47.435746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:47.435746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:47.435746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:47.435746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:47.435746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:47.435746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:47.437012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:47.437012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:47.437012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:47.437012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:47.437012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:47.437012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:47.437012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:47.437012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:47.437012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:47.437012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:47.437012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:47.437012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:47.444913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 274/552] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:47.450697 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553216e6-e2db-4088-929a-bc5b180a256e None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:47.488980 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:47.562667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13155767-c0ff-4122-8743-1fc517bde764 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:47.623441 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109c713-a16c-4c20-87e4-088cf0cd5a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:47.624068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109c713-a16c-4c20-87e4-088cf0cd5a37 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:47.624392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109c713-a16c-4c20-87e4-088cf0cd5a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:47.624686 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109c713-a16c-4c20-87e4-088cf0cd5a37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:47.657007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553216e6-e2db-4088-929a-bc5b180a256e None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:47.706571 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:47.720538 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109c713-a16c-4c20-87e4-088cf0cd5a37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:47.722357 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b5324-d42a-4594-bf90-2b3d4d1432a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:47.722830 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109c713-a16c-4c20-87e4-088cf0cd5a37 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:47.723638 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b5324-d42a-4594-bf90-2b3d4d1432a1 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:47.724110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b5324-d42a-4594-bf90-2b3d4d1432a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:47.724675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b5324-d42a-4594-bf90-2b3d4d1432a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:47.725689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b5324-d42a-4594-bf90-2b3d4d1432a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:47.726200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b5324-d42a-4594-bf90-2b3d4d1432a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:47.727757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b5324-d42a-4594-bf90-2b3d4d1432a1 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:47.737020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 275/553] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:04:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:47.739222 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3f600-8ce4-45eb-8694-31dd0e17bad9 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:47.739703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3f600-8ce4-45eb-8694-31dd0e17bad9 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:47.740189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3f600-8ce4-45eb-8694-31dd0e17bad9 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:47.742065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 276/554] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:47.742960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3f600-8ce4-45eb-8694-31dd0e17bad9 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:47.838516 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 279/555] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 02:04:47.839205 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:47.841401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76113c75-644b-4f7c-859a-83b4249b8640 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:47.842083 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76113c75-644b-4f7c-859a-83b4249b8640 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:47.842590 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76113c75-644b-4f7c-859a-83b4249b8640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:47.866091 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-76113c75-644b-4f7c-859a-83b4249b8640 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:47.867680 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-76113c75-644b-4f7c-859a-83b4249b8640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:47.873193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3f600-8ce4-45eb-8694-31dd0e17bad9 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:47.915915 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc17fd-eab0-449f-9b6f-780bb7a37f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:47.916570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc17fd-eab0-449f-9b6f-780bb7a37f10 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:47.916942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc17fd-eab0-449f-9b6f-780bb7a37f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:47.917223 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc17fd-eab0-449f-9b6f-780bb7a37f10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:47.931425 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cc17fd-eab0-449f-9b6f-780bb7a37f10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:47.933431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cc17fd-eab0-449f-9b6f-780bb7a37f10 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:47.950003 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 280/556] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:47.950998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-39cc17fd-eab0-449f-9b6f-780bb7a37f10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:47.950998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:47.950998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:47.950998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:47.950998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:47.950998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:47.950998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:47.950998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:47.950998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:47.950998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:47.950998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:47.950998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:47.950998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:47.950998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:47.950998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:47.951852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:47.951852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:47.951852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:47.951852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:47.951852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:47.951852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:47.951852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:47.951852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:47.951852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:47.951852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:47.951852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:47.951852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:47.951852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:47.951852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:47.951852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:47.951852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:47.951852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:47.952751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:47.952751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:47.952751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:47.952751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:47.952751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 277/557] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:47.953658 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76113c75-644b-4f7c-859a-83b4249b8640 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:47.964719 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc17fd-eab0-449f-9b6f-780bb7a37f10 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:47.984330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:48.082125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76113c75-644b-4f7c-859a-83b4249b8640 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:48.100070 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc17fd-eab0-449f-9b6f-780bb7a37f10 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:48.123411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b642261-bcfb-4f50-a6d9-edade380d028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:48.124571 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b642261-bcfb-4f50-a6d9-edade380d028 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:48.125234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:48.126315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b642261-bcfb-4f50-a6d9-edade380d028 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:48.126788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b642261-bcfb-4f50-a6d9-edade380d028 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:48.136763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23b51f-3c58-45a5-90a4-b95773d398c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:48.137381 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23b51f-3c58-45a5-90a4-b95773d398c3 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:48.137660 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23b51f-3c58-45a5-90a4-b95773d398c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:48.137931 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b23b51f-3c58-45a5-90a4-b95773d398c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:48.138851 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23b51f-3c58-45a5-90a4-b95773d398c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:48.139037 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23b51f-3c58-45a5-90a4-b95773d398c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:48.140413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b23b51f-3c58-45a5-90a4-b95773d398c3 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:48.141295 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b642261-bcfb-4f50-a6d9-edade380d028 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:48.143179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b642261-bcfb-4f50-a6d9-edade380d028 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:48.153366 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 278/558] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:48.154281 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23b51f-3c58-45a5-90a4-b95773d398c3 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:48.160933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-6b642261-bcfb-4f50-a6d9-edade380d028 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:48.160933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:48.160933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:48.160933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:48.160933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:48.160933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:48.160933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:48.160933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:48.160933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:48.160933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:48.160933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:48.160933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:48.160933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:48.160933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:48.160933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:48.162304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:48.162304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:48.162304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:48.162304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:48.162304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:48.162304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:48.162304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:48.162304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:48.162304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:48.162304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:48.162304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:48.162304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:48.162304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:48.162304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:48.162304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:48.162304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:48.162304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:48.163162 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:48.163162 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:48.163162 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:48.163162 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:48.169891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 281/559] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:48.171132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b642261-bcfb-4f50-a6d9-edade380d028 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:48.253307 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:48.263280 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b23b51f-3c58-45a5-90a4-b95773d398c3 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:48.290785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b642261-bcfb-4f50-a6d9-edade380d028 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:48.302710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6a5e5-eb08-42ce-bed4-bf149a8fd913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:48.303567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6a5e5-eb08-42ce-bed4-bf149a8fd913 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:48.304078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6a5e5-eb08-42ce-bed4-bf149a8fd913 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:48.304487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6a5e5-eb08-42ce-bed4-bf149a8fd913 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:48.317273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6a5e5-eb08-42ce-bed4-bf149a8fd913 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:48.322773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-9cb6a5e5-eb08-42ce-bed4-bf149a8fd913 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:48.322773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:48.322773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:48.322773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:48.322773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:48.322773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:48.322773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:48.322773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:48.322773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:48.322773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:48.322773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:48.322773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:48.322773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:48.322773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:48.322773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:48.322773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:48.323860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:48.323860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:48.323860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:48.323860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:48.323860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:48.323860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:48.323860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:48.323860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:48.323860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:48.323860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:48.323860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:48.323860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:48.323860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:48.323860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:48.323860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:48.323860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:48.324973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:48.324973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:48.324973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:48.324973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:48.324973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:48.324973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:48.324973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:48.324973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:48.324973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:48.324973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:48.324973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:48.324973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:48.327401 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 279/560] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:48.328565 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6a5e5-eb08-42ce-bed4-bf149a8fd913 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:48.333620 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ebcba-e595-4490-9832-f0a7d5f0df8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:48.334061 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ebcba-e595-4490-9832-f0a7d5f0df8e None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:48.334407 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ebcba-e595-4490-9832-f0a7d5f0df8e None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:48.334782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ebcba-e595-4490-9832-f0a7d5f0df8e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:48.343707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ebcba-e595-4490-9832-f0a7d5f0df8e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:48.345432 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ebcba-e595-4490-9832-f0a7d5f0df8e None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:48.358129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-449ebcba-e595-4490-9832-f0a7d5f0df8e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:04:48.358129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:48.358129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:48.358129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:48.358129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:48.358129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:48.358129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:48.358129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:48.358129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:48.358129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:48.358129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:48.358129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:48.358129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:04:48.358129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:04:48.358129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:04:48.358129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:48.359090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:48.360753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:48.360753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:48.360753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:04:48.360753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:04:48.360753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:04:48.360753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:04:48.360753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:04:48.360753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:04:48.360753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:04:48.360753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:48.360753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 282/561] 10.222.0.45 () {56 vars in 1093 bytes} [Thu Aug 4 02:04:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:48.360753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ebcba-e595-4490-9832-f0a7d5f0df8e None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:48.385747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:48.444782 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6a5e5-eb08-42ce-bed4-bf149a8fd913 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:48.470542 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ebcba-e595-4490-9832-f0a7d5f0df8e None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:48.490809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e2d4f-a4b8-45c9-81d8-22d44ed02d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:48.491429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e2d4f-a4b8-45c9-81d8-22d44ed02d3f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:48.491716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e2d4f-a4b8-45c9-81d8-22d44ed02d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:48.491991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e2d4f-a4b8-45c9-81d8-22d44ed02d3f None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:48.492717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e2d4f-a4b8-45c9-81d8-22d44ed02d3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:48.493024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e2d4f-a4b8-45c9-81d8-22d44ed02d3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:48.494403 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e2d4f-a4b8-45c9-81d8-22d44ed02d3f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:48.506519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 280/562] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:48.507738 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b87a8-1eb4-419d-adff-541a5acbfb2a None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:48.508027 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b87a8-1eb4-419d-adff-541a5acbfb2a None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:48.508298 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b87a8-1eb4-419d-adff-541a5acbfb2a None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:48.512663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 281/563] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:48.512663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b87a8-1eb4-419d-adff-541a5acbfb2a None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:48.513156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:48.516031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228b8a86-074a-4ba0-a64a-791ca38a3def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:48.516671 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b8a86-074a-4ba0-a64a-791ca38a3def None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:48.516956 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b8a86-074a-4ba0-a64a-791ca38a3def None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:48.517225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228b8a86-074a-4ba0-a64a-791ca38a3def None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:48.517949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b8a86-074a-4ba0-a64a-791ca38a3def None admin] RBAC: Adding query filter params (name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:48.518250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b8a86-074a-4ba0-a64a-791ca38a3def None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:48.519656 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228b8a86-074a-4ba0-a64a-791ca38a3def None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:48.532476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 283/564] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:04:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:48.532476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78a379-354b-4eab-9b82-a25f67f09750 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:48.532476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78a379-354b-4eab-9b82-a25f67f09750 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:48.532476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c78a379-354b-4eab-9b82-a25f67f09750 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:48.532476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 284/565] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:48.532476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78a379-354b-4eab-9b82-a25f67f09750 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:48.618354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b87a8-1eb4-419d-adff-541a5acbfb2a None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:48.620786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:48.638936 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c78a379-354b-4eab-9b82-a25f67f09750 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:48.652028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfd56f-6f53-4e2f-b3fb-b5012f3e88ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:48.652831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfd56f-6f53-4e2f-b3fb-b5012f3e88ff None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:48.653112 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfd56f-6f53-4e2f-b3fb-b5012f3e88ff None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:48.653388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfd56f-6f53-4e2f-b3fb-b5012f3e88ff None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:48.654190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfd56f-6f53-4e2f-b3fb-b5012f3e88ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:48.654497 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfd56f-6f53-4e2f-b3fb-b5012f3e88ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:48.655932 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfd56f-6f53-4e2f-b3fb-b5012f3e88ff None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:48.673436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 282/566] 10.222.0.45 () {56 vars in 1133 bytes} [Thu Aug 4 02:04:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:48.674656 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4d2f1-b6a0-4479-8f17-b1bf1bf6349f None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:48.675505 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4d2f1-b6a0-4479-8f17-b1bf1bf6349f None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:48.675777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e4d2f1-b6a0-4479-8f17-b1bf1bf6349f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:48.676330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb235fb-275f-44f5-be5e-f49d961fea8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:48.676989 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb235fb-275f-44f5-be5e-f49d961fea8d None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:48.677271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb235fb-275f-44f5-be5e-f49d961fea8d None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:48.677539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb235fb-275f-44f5-be5e-f49d961fea8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:48.684392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb235fb-275f-44f5-be5e-f49d961fea8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:48.685837 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb235fb-275f-44f5-be5e-f49d961fea8d None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:48.693917 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-d1e4d2f1-b6a0-4479-8f17-b1bf1bf6349f None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:48.695099 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d1e4d2f1-b6a0-4479-8f17-b1bf1bf6349f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:48.697132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-cfb235fb-275f-44f5-be5e-f49d961fea8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:48.697132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:48.697132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:48.697132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:48.697132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:48.697132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:48.697132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:48.697132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:48.697132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:48.697132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:48.697132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:48.697132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:48.697132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:04:48.697132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:04:48.697132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:04:48.698065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:04:48.698065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:48.698065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:48.698065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:48.698065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:48.698065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:48.698065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:48.698065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:48.698065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:48.698065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:48.698065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:48.698065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:48.698065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:48.698065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:48.698065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:48.698065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:48.698065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:48.698961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:04:48.698961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:04:48.698961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:04:48.698961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:04:48.698961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:04:48.698961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:04:48.698961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:48.698961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:48.698961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 285/567] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:48.699449 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb235fb-275f-44f5-be5e-f49d961fea8d None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:48.755076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:48.763018 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 283/568] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:48.764404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f4522-e5f2-4bfc-82e3-6f122325baf7 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:48.764723 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f4522-e5f2-4bfc-82e3-6f122325baf7 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:48.765001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f4522-e5f2-4bfc-82e3-6f122325baf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:48.786566 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-9d3f4522-e5f2-4bfc-82e3-6f122325baf7 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:48.787751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9d3f4522-e5f2-4bfc-82e3-6f122325baf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:48.802315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb235fb-275f-44f5-be5e-f49d961fea8d None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:48.835013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57439b-4d0a-441c-b77c-e6e0293d2627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:48.835643 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57439b-4d0a-441c-b77c-e6e0293d2627 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:48.835926 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57439b-4d0a-441c-b77c-e6e0293d2627 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:48.836199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f57439b-4d0a-441c-b77c-e6e0293d2627 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:48.847140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57439b-4d0a-441c-b77c-e6e0293d2627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:48.848575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f57439b-4d0a-441c-b77c-e6e0293d2627 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:48.854360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 284/569] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:48.855195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f4522-e5f2-4bfc-82e3-6f122325baf7 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:48.878140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-0f57439b-4d0a-441c-b77c-e6e0293d2627 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:48.878140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:48.878140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:48.878140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:48.878140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:48.878140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:48.878140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:48.878140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:48.878140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:48.878140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:48.878140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:48.878140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:48.878140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:48.878140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:48.878140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:48.879151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:48.879151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:48.879151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:48.879151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:48.879151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:48.879151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:48.879151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:48.879151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:48.879151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:48.879151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:48.879151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:48.879151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:48.879151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:48.879151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:48.879151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:48.879151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:48.879151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:48.880017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:48.880017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:48.880017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:48.880017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:48.880244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:48.880315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 286/570] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:48.880315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57439b-4d0a-441c-b77c-e6e0293d2627 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:48.959309 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f4522-e5f2-4bfc-82e3-6f122325baf7 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:48.983281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f57439b-4d0a-441c-b77c-e6e0293d2627 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:48.990040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:48.992725 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c4069-e8e3-4971-b07b-c65adb9f2c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:48.993356 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c4069-e8e3-4971-b07b-c65adb9f2c97 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:48.993629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c4069-e8e3-4971-b07b-c65adb9f2c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:48.993893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c4069-e8e3-4971-b07b-c65adb9f2c97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:49.003868 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c4069-e8e3-4971-b07b-c65adb9f2c97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:49.005258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c4069-e8e3-4971-b07b-c65adb9f2c97 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:49.017638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21200351-acd0-4ef2-9e4e-e36ad6c190f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:49.018246 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21200351-acd0-4ef2-9e4e-e36ad6c190f2 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:49.019037 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21200351-acd0-4ef2-9e4e-e36ad6c190f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:49.019304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21200351-acd0-4ef2-9e4e-e36ad6c190f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:49.020035 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21200351-acd0-4ef2-9e4e-e36ad6c190f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:49.020152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-085c4069-e8e3-4971-b07b-c65adb9f2c97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:49.020152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:49.020152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:49.020152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:49.020152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:49.020152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:49.020152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:49.020152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:49.020152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:49.020152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:49.020152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:49.020152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:49.020152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:49.020152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:49.020152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:49.021016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:49.021016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:49.021016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:49.021016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:49.021016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:49.021016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:49.021016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:49.021016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:49.021016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:49.021016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:49.021016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:49.021016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:49.021016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:49.021016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:49.021016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:49.021016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:49.021016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:49.022188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:49.022188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:49.022188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:49.022188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:49.022567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21200351-acd0-4ef2-9e4e-e36ad6c190f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:49.022643 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 285/571] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:49.022711 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21200351-acd0-4ef2-9e4e-e36ad6c190f2 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:49.022777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c4069-e8e3-4971-b07b-c65adb9f2c97 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:49.033658 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 287/572] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:04:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:49.035074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21200351-acd0-4ef2-9e4e-e36ad6c190f2 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:49.098352 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:49.123613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c4069-e8e3-4971-b07b-c65adb9f2c97 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:49.133171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21200351-acd0-4ef2-9e4e-e36ad6c190f2 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:49.156910 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacadde4-b47d-4020-85bb-dd399b2e35a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:49.157537 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacadde4-b47d-4020-85bb-dd399b2e35a2 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:49.157815 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacadde4-b47d-4020-85bb-dd399b2e35a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:49.158081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacadde4-b47d-4020-85bb-dd399b2e35a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:49.167808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e33a37-1fe8-4aa0-8d33-56020197e8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:49.168445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e33a37-1fe8-4aa0-8d33-56020197e8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:49.168772 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e33a37-1fe8-4aa0-8d33-56020197e8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:49.168869 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacadde4-b47d-4020-85bb-dd399b2e35a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:49.169049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e33a37-1fe8-4aa0-8d33-56020197e8e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:49.174170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-bacadde4-b47d-4020-85bb-dd399b2e35a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:49.174170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:49.174170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:49.174170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:49.174170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:49.174170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:49.174170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:49.174170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:49.174170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:49.174170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:49.174170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:49.174170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:49.174170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:49.174170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:49.174170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:49.174170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:49.175301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:49.175301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:49.175301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:49.175301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:49.175301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:49.175301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:49.175301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:49.175301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:49.175301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:49.175301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:49.175301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:49.175301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:49.175301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:49.175301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:49.175301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:49.175301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:49.176318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:49.176318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:49.176318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:49.176318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:49.176318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:49.176318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:49.176318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:49.176318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:49.176318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:49.176318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:49.176318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:49.176318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:49.176318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 286/573] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:49.176318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacadde4-b47d-4020-85bb-dd399b2e35a2 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:49.182519 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e33a37-1fe8-4aa0-8d33-56020197e8e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:49.187713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-67e33a37-1fe8-4aa0-8d33-56020197e8e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:49.187713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:49.187713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:49.187713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:49.187713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:49.187713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:49.187713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:49.187713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:49.187713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:49.187713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:49.187713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:49.187713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:49.187713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:49.187713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:49.187713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:49.187713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:49.188865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:49.188865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:49.188865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:49.188865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:49.188865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:49.188865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:49.188865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:49.188865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:49.188865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:49.188865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:49.188865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:49.188865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:49.188865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:49.188865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:49.188865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:49.188865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:49.194603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:49.194603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:49.194603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:49.194603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:49.194603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:49.194603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:49.194603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:49.194603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:49.194603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:49.194603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:49.194603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:49.194603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:49.194603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 288/574] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:49.194603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e33a37-1fe8-4aa0-8d33-56020197e8e8 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:49.218418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:49.277834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacadde4-b47d-4020-85bb-dd399b2e35a2 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:49.290862 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e33a37-1fe8-4aa0-8d33-56020197e8e8 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:49.312455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790a303-45dc-485f-998f-85c702ed1e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:49.313109 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790a303-45dc-485f-998f-85c702ed1e76 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:49.313397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790a303-45dc-485f-998f-85c702ed1e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:49.313674 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790a303-45dc-485f-998f-85c702ed1e76 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:49.314366 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790a303-45dc-485f-998f-85c702ed1e76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:49.314669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790a303-45dc-485f-998f-85c702ed1e76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:49.316568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790a303-45dc-485f-998f-85c702ed1e76 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:49.318010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:49.325266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de15aa-cb34-405d-b078-50cf089b52fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:49.325888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de15aa-cb34-405d-b078-50cf089b52fd None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:49.326161 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de15aa-cb34-405d-b078-50cf089b52fd None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:49.326438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de15aa-cb34-405d-b078-50cf089b52fd None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:49.327127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de15aa-cb34-405d-b078-50cf089b52fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:49.327430 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de15aa-cb34-405d-b078-50cf089b52fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:49.328808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de15aa-cb34-405d-b078-50cf089b52fd None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:49.329120 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 287/575] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:49.329920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790a303-45dc-485f-998f-85c702ed1e76 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:49.348929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 289/576] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:49.348929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de15aa-cb34-405d-b078-50cf089b52fd None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:49.430120 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790a303-45dc-485f-998f-85c702ed1e76 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:49.432116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:49.448234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de15aa-cb34-405d-b078-50cf089b52fd None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:49.464734 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d9f12-8831-4afb-83f2-2248a418209b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:49.465357 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d9f12-8831-4afb-83f2-2248a418209b None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:49.465635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d9f12-8831-4afb-83f2-2248a418209b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:49.465912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d9f12-8831-4afb-83f2-2248a418209b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:49.467144 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d9f12-8831-4afb-83f2-2248a418209b None admin] RBAC: Adding query filter params (role.id=977824a8c6a24c00af2f6dabd0e2a6f0, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=155a7276efab426cb34175683c475f33) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:49.467144 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d9f12-8831-4afb-83f2-2248a418209b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:49.468431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31d9f12-8831-4afb-83f2-2248a418209b None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:49.474103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 288/577] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:49 2022] GET /identity/v3/role_assignments?user.id=155a7276efab426cb34175683c475f33&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=977824a8c6a24c00af2f6dabd0e2a6f0 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:49.474881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d9f12-8831-4afb-83f2-2248a418209b None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:49.491478 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fb452-109c-49c2-8b2f-4011f1b98031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:49.492779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fb452-109c-49c2-8b2f-4011f1b98031 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:49.493062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fb452-109c-49c2-8b2f-4011f1b98031 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:49.493337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fb452-109c-49c2-8b2f-4011f1b98031 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:49.494085 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6fb452-109c-49c2-8b2f-4011f1b98031 None admin] RBAC: Adding query filter params (name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:49.494390 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6fb452-109c-49c2-8b2f-4011f1b98031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:49.496331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6fb452-109c-49c2-8b2f-4011f1b98031 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:49.512510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 290/578] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:04:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:49.512510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fb452-109c-49c2-8b2f-4011f1b98031 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:49.550499 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:49.600770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d9f12-8831-4afb-83f2-2248a418209b None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:49.635537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fb452-109c-49c2-8b2f-4011f1b98031 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:49.638622 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d37fb-224f-4656-a50f-e35ba69fcfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:49.639267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d37fb-224f-4656-a50f-e35ba69fcfdd None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:49.639541 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d37fb-224f-4656-a50f-e35ba69fcfdd None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:49.639812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12d37fb-224f-4656-a50f-e35ba69fcfdd None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:49.640585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d37fb-224f-4656-a50f-e35ba69fcfdd None admin] RBAC: Adding query filter params (name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:49.640924 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d37fb-224f-4656-a50f-e35ba69fcfdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:49.642319 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12d37fb-224f-4656-a50f-e35ba69fcfdd None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:49.659143 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 289/579] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:04:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:49.660021 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d37fb-224f-4656-a50f-e35ba69fcfdd None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:49.682211 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd7c5a-5301-43c9-8f9f-b22b2daf227c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:49.682837 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd7c5a-5301-43c9-8f9f-b22b2daf227c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:49.683026 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd7c5a-5301-43c9-8f9f-b22b2daf227c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:49.683293 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd7c5a-5301-43c9-8f9f-b22b2daf227c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:49.691923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd7c5a-5301-43c9-8f9f-b22b2daf227c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:49.693392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd7c5a-5301-43c9-8f9f-b22b2daf227c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:49.704773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-2cbd7c5a-5301-43c9-8f9f-b22b2daf227c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 02:04:49.704773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:49.704773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:49.704773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:49.704773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:49.704773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:49.704773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:49.704773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:49.704773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:49.704773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:49.704773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:49.704773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:49.704773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:04:49.704773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:04:49.704773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:04:49.704773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:49.705743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:49.707281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:49.707281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:49.707281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:04:49.707281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:04:49.707281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:04:49.707281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:04:49.707281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:04:49.707281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:04:49.707281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 02:04:49.707281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:49.707281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 291/580] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:49.707281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd7c5a-5301-43c9-8f9f-b22b2daf227c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:49.722411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:49.776892 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12d37fb-224f-4656-a50f-e35ba69fcfdd None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:49.797914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:49.813973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b891583-7522-4058-bde7-c8fb78d35928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:49.814622 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b891583-7522-4058-bde7-c8fb78d35928 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:49.814878 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd7c5a-5301-43c9-8f9f-b22b2daf227c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:49.814983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b891583-7522-4058-bde7-c8fb78d35928 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:49.815257 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b891583-7522-4058-bde7-c8fb78d35928 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:49.822805 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b891583-7522-4058-bde7-c8fb78d35928 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:49.824267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b891583-7522-4058-bde7-c8fb78d35928 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:49.838326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-0b891583-7522-4058-bde7-c8fb78d35928 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:49.838326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:49.838326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:49.838326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:49.838326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:49.838326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:49.838326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:49.838326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:49.838326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:49.838326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:49.838326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:49.838326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:49.838326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:04:49.838326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:04:49.838326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:04:49.839245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:04:49.839245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:49.839245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:49.839245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:49.839245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:49.839245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:49.839245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:49.839245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:49.839245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:49.839245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:49.839245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:49.839245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:49.839245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:49.839245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:49.839245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:49.839245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:49.839245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:49.840209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:04:49.840209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:04:49.840209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:04:49.840209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:04:49.840209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:04:49.840209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:04:49.840209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:49.840209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:49.840209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 290/581] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:49.840209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b891583-7522-4058-bde7-c8fb78d35928 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:49.856669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8054942-0639-4bf2-9b8a-63ac5af0b283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:49.857598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8054942-0639-4bf2-9b8a-63ac5af0b283 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:49.857598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8054942-0639-4bf2-9b8a-63ac5af0b283 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:49.857833 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8054942-0639-4bf2-9b8a-63ac5af0b283 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:49.869466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8054942-0639-4bf2-9b8a-63ac5af0b283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:49.871011 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8054942-0639-4bf2-9b8a-63ac5af0b283 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:49.885864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-c8054942-0639-4bf2-9b8a-63ac5af0b283 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:49.885864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:49.885864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:49.885864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:49.885864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:49.885864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:49.885864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:49.885864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:49.885864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:49.885864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:49.885864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:49.885864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:49.885864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:49.885864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:49.885864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:49.886788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:49.886788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:49.886788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:49.886788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:49.886788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:49.886788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:49.886788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:49.886788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:49.886788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:49.886788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:49.886788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:49.886788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:49.886788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:49.886788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:49.886788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:49.886788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:49.886788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:49.887678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:49.887678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:49.887678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:49.887678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:49.891761 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 292/582] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:49.892463 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8054942-0639-4bf2-9b8a-63ac5af0b283 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:49.956461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:49.967550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b891583-7522-4058-bde7-c8fb78d35928 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:50.000893 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8054942-0639-4bf2-9b8a-63ac5af0b283 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:50.005998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e3861-0db6-46fd-ba70-eb70244a2fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:50.006636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e3861-0db6-46fd-ba70-eb70244a2fbf None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:50.006913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e3861-0db6-46fd-ba70-eb70244a2fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:50.007184 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147e3861-0db6-46fd-ba70-eb70244a2fbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:50.018036 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e3861-0db6-46fd-ba70-eb70244a2fbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:50.020017 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147e3861-0db6-46fd-ba70-eb70244a2fbf None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:50.034831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-147e3861-0db6-46fd-ba70-eb70244a2fbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:50.034831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:50.034831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:50.034831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:50.034831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:50.034831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:50.034831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:50.034831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:50.034831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:50.034831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:50.034831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:50.034831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:50.034831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:50.034831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:50.034831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:50.035771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:50.035771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:50.035771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:50.035771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:50.035771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:50.035771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:50.035771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:50.035771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:50.035771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:50.035771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:50.035771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:50.035771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:50.035771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:50.035771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:50.035771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:50.035771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:50.035771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:50.036749 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:50.036749 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:50.036749 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:50.036749 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:50.036749 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 291/583] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:50.037041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e3861-0db6-46fd-ba70-eb70244a2fbf None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:50.044598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1e545-1410-4859-b35e-56fc38e8d9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:50.045267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1e545-1410-4859-b35e-56fc38e8d9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:50.045547 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1e545-1410-4859-b35e-56fc38e8d9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:50.045821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1e545-1410-4859-b35e-56fc38e8d9a9 None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:50.046626 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1e545-1410-4859-b35e-56fc38e8d9a9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:50.046931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1e545-1410-4859-b35e-56fc38e8d9a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:50.048352 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1e545-1410-4859-b35e-56fc38e8d9a9 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:50.071956 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 293/584] 10.222.0.45 () {56 vars in 1137 bytes} [Thu Aug 4 02:04:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:50.071956 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1e545-1410-4859-b35e-56fc38e8d9a9 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:50.076432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:50.154070 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147e3861-0db6-46fd-ba70-eb70244a2fbf None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:50.181362 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1e545-1410-4859-b35e-56fc38e8d9a9 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:50.190699 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440bd781-1a27-4386-bee6-78c59d6f4d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:50.191318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bd781-1a27-4386-bee6-78c59d6f4d8b None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:50.191600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bd781-1a27-4386-bee6-78c59d6f4d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:50.191876 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bd781-1a27-4386-bee6-78c59d6f4d8b None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:50.192646 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440bd781-1a27-4386-bee6-78c59d6f4d8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:50.192959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440bd781-1a27-4386-bee6-78c59d6f4d8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:50.194397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440bd781-1a27-4386-bee6-78c59d6f4d8b None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:50.201145 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:50.208021 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 292/585] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:04:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:50.208608 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440bd781-1a27-4386-bee6-78c59d6f4d8b None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:50.225399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8488e-769d-4cfa-89ae-2a0db539b17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:50.226110 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8488e-769d-4cfa-89ae-2a0db539b17a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:50.226457 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8488e-769d-4cfa-89ae-2a0db539b17a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:50.226795 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8488e-769d-4cfa-89ae-2a0db539b17a None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:50.227558 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8488e-769d-4cfa-89ae-2a0db539b17a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:50.227930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8488e-769d-4cfa-89ae-2a0db539b17a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:50.229388 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8488e-769d-4cfa-89ae-2a0db539b17a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:50.242241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 294/586] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:50.242987 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8488e-769d-4cfa-89ae-2a0db539b17a None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:50.309753 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:50.319811 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440bd781-1a27-4386-bee6-78c59d6f4d8b None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:50.350085 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8488e-769d-4cfa-89ae-2a0db539b17a None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:50.357795 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6385408-b201-4a37-bacc-a2967e38f53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:50.358509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6385408-b201-4a37-bacc-a2967e38f53e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:50.358860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6385408-b201-4a37-bacc-a2967e38f53e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:50.359204 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6385408-b201-4a37-bacc-a2967e38f53e None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:50.359971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6385408-b201-4a37-bacc-a2967e38f53e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:50.360342 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6385408-b201-4a37-bacc-a2967e38f53e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:50.361818 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6385408-b201-4a37-bacc-a2967e38f53e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:50.378241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 293/587] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:50.378241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6385408-b201-4a37-bacc-a2967e38f53e None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:50.390954 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df49522-7a2a-40c4-a6b5-8920f17de051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:50.391699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df49522-7a2a-40c4-a6b5-8920f17de051 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:50.392041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df49522-7a2a-40c4-a6b5-8920f17de051 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:50.392382 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df49522-7a2a-40c4-a6b5-8920f17de051 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:50.403560 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df49522-7a2a-40c4-a6b5-8920f17de051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:50.405104 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df49522-7a2a-40c4-a6b5-8920f17de051 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:50.419669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-1df49522-7a2a-40c4-a6b5-8920f17de051 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:50.419669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:50.419669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:50.419669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:50.419669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:50.419669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:50.419669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:50.419669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:50.419669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:50.419669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:50.419669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:50.419669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:50.419669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:50.419669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:50.419669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:50.420808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:50.420808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:50.420808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:50.420808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:50.420808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:50.420808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:50.420808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:50.420808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:50.420808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:50.420808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:50.420808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:50.420808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:50.420808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:50.420808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:50.420808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:50.420808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:50.420808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:50.421925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:50.421925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:50.421925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:50.421925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:50.427488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 295/588] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:50.427488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df49522-7a2a-40c4-a6b5-8920f17de051 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:50.453911 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:50.503960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6385408-b201-4a37-bacc-a2967e38f53e None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:50.528763 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df49522-7a2a-40c4-a6b5-8920f17de051 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:50.542804 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d0236-8508-4049-8772-78298930bcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:50.542804 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d0236-8508-4049-8772-78298930bcb0 None admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:50.542804 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d0236-8508-4049-8772-78298930bcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:50.543189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d0236-8508-4049-8772-78298930bcb0 None admin] PATH_INFO: `/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/c571406d2a3f40dab035a918399d6f6a/roles/977824a8c6a24c00af2f6dabd0e2a6f0` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:50.558831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d0236-8508-4049-8772-78298930bcb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b8caff828a4f5289094fb2ae2278f7, user_id=c571406d2a3f40dab035a918399d6f6a, role_id=977824a8c6a24c00af2f6dabd0e2a6f0)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:50.560270 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d0236-8508-4049-8772-78298930bcb0 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:50.569716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ff2b1-f1ad-42e0-90fb-3208b3b234d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:50.570359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ff2b1-f1ad-42e0-90fb-3208b3b234d6 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:50.570642 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ff2b1-f1ad-42e0-90fb-3208b3b234d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:50.570919 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ff2b1-f1ad-42e0-90fb-3208b3b234d6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:50.573961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 294/589] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:04:50 2022] PUT /identity/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/c571406d2a3f40dab035a918399d6f6a/roles/977824a8c6a24c00af2f6dabd0e2a6f0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:04:50.574975 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d0236-8508-4049-8772-78298930bcb0 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:50.577828 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:50.587053 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ff2b1-f1ad-42e0-90fb-3208b3b234d6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:50.588599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ff2b1-f1ad-42e0-90fb-3208b3b234d6 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:50.602716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-ec6ff2b1-f1ad-42e0-90fb-3208b3b234d6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 02:04:50.602716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:50.602716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:50.602716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:50.602716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:50.602716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:50.602716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:50.602716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:50.602716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:50.602716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:50.602716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:50.602716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:50.602716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:04:50.602716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:04:50.602716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:04:50.602716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:50.603920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:50.605219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:50.605219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:50.605219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:04:50.605219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:04:50.605219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:04:50.605219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:04:50.605219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:04:50.605219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:04:50.605219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 02:04:50.605219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:50.608926 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 296/590] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:50.609895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ff2b1-f1ad-42e0-90fb-3208b3b234d6 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:50.727400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:50.739821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ff2b1-f1ad-42e0-90fb-3208b3b234d6 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:50.741467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d0236-8508-4049-8772-78298930bcb0 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:50.781799 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f1640-36b7-48b1-a65a-6879d9efcb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:50.782345 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f1640-36b7-48b1-a65a-6879d9efcb1b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:50.782627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f1640-36b7-48b1-a65a-6879d9efcb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:50.782901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f1640-36b7-48b1-a65a-6879d9efcb1b None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:50.783593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f1640-36b7-48b1-a65a-6879d9efcb1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:50.783898 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f1640-36b7-48b1-a65a-6879d9efcb1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:50.785329 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ffe3a-c4c2-47b2-8a73-87aa12f9dcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:50.786016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6f1640-36b7-48b1-a65a-6879d9efcb1b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:50.786755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ffe3a-c4c2-47b2-8a73-87aa12f9dcdf None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:50.787233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ffe3a-c4c2-47b2-8a73-87aa12f9dcdf None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:50.787989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ffe3a-c4c2-47b2-8a73-87aa12f9dcdf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:50.798098 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ffe3a-c4c2-47b2-8a73-87aa12f9dcdf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:50.798934 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 297/591] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:50.799835 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f1640-36b7-48b1-a65a-6879d9efcb1b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:50.800401 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ffe3a-c4c2-47b2-8a73-87aa12f9dcdf None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:50.819474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-c45ffe3a-c4c2-47b2-8a73-87aa12f9dcdf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 02:04:50.819474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:50.819474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:50.819474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:50.819474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:50.819474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:50.819474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:50.819474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:50.819474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:50.819474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:50.819474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:50.819474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:50.819474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:04:50.819474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:04:50.819474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:04:50.819474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:50.820464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:50.821450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:50.821450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:50.821450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:04:50.821450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:04:50.821450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:04:50.821450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:04:50.821450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:04:50.821450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:04:50.821450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 02:04:50.821450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:50.821450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 295/592] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:50.822047 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ffe3a-c4c2-47b2-8a73-87aa12f9dcdf None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:50.897222 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:50.935060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ffe3a-c4c2-47b2-8a73-87aa12f9dcdf None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:50.939376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6f1640-36b7-48b1-a65a-6879d9efcb1b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:50.978035 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597726ed-c331-4dff-a901-364e5760d4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:50.979032 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597726ed-c331-4dff-a901-364e5760d4eb None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:50.979541 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597726ed-c331-4dff-a901-364e5760d4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:50.980015 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597726ed-c331-4dff-a901-364e5760d4eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:50.982886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04257f94-4f24-4189-b343-3d25366747ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:50.983507 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04257f94-4f24-4189-b343-3d25366747ee None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:50.983785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04257f94-4f24-4189-b343-3d25366747ee None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:50.984114 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04257f94-4f24-4189-b343-3d25366747ee None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:50.984943 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04257f94-4f24-4189-b343-3d25366747ee None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:50.985247 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04257f94-4f24-4189-b343-3d25366747ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:50.986674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04257f94-4f24-4189-b343-3d25366747ee None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:50.989880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597726ed-c331-4dff-a901-364e5760d4eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:50.991541 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597726ed-c331-4dff-a901-364e5760d4eb None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:51.003887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-597726ed-c331-4dff-a901-364e5760d4eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:51.003887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:51.003887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:51.003887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:51.003887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:51.003887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:51.003887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:51.003887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:51.003887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:51.003887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:51.003887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:51.003887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:51.003887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:04:51.003887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:04:51.003887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:04:51.004858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:04:51.004858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:51.004858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:51.004858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:51.004858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:51.004858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:51.004858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:51.004858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:51.004858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:51.004858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:51.004858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:51.004858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:51.004858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:51.004858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:51.004858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:51.004858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:51.004858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:51.005761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:04:51.005761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:04:51.005761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:04:51.005761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:04:51.005761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:04:51.005761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:04:51.005761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:51.005761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:51.012426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 298/594] 10.222.0.45 () {56 vars in 1137 bytes} [Thu Aug 4 02:04:50 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:51.012426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04257f94-4f24-4189-b343-3d25366747ee None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:51.012843 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 296/594] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:51.014010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597726ed-c331-4dff-a901-364e5760d4eb None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:51.022134 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:51.118513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04257f94-4f24-4189-b343-3d25366747ee None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:51.129016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597726ed-c331-4dff-a901-364e5760d4eb None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:51.154053 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e3d28-6118-4f37-941b-f69d5f26cc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:51.154716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e3d28-6118-4f37-941b-f69d5f26cc28 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:51.155009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e3d28-6118-4f37-941b-f69d5f26cc28 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:51.155471 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e3d28-6118-4f37-941b-f69d5f26cc28 None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:51.156059 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e3d28-6118-4f37-941b-f69d5f26cc28 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:51.156362 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e3d28-6118-4f37-941b-f69d5f26cc28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:51.157822 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e3d28-6118-4f37-941b-f69d5f26cc28 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:51.164950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:51.169410 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298e3f1-54c6-4a02-830b-c361e09a9e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:51.170297 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298e3f1-54c6-4a02-830b-c361e09a9e88 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:51.170775 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298e3f1-54c6-4a02-830b-c361e09a9e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:51.171253 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298e3f1-54c6-4a02-830b-c361e09a9e88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:51.183103 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 299/595] 10.222.0.45 () {56 vars in 1139 bytes} [Thu Aug 4 02:04:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:51.184018 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e3d28-6118-4f37-941b-f69d5f26cc28 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:51.184237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298e3f1-54c6-4a02-830b-c361e09a9e88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:51.185926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298e3f1-54c6-4a02-830b-c361e09a9e88 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:51.202186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-f298e3f1-54c6-4a02-830b-c361e09a9e88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:51.202186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:51.202186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:51.202186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:51.202186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:51.202186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:51.202186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:51.202186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:51.202186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:51.202186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:51.202186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:51.202186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:51.202186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:51.202186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:51.202186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:51.203189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:51.203189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:51.203189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:51.203189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:51.203189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:51.203189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:51.203189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:51.203189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:51.203189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:51.203189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:51.203189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:51.203189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:51.203189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:51.203189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:51.203189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:51.203189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:51.203189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:51.204075 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:51.204075 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:51.204075 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:51.204075 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:51.206771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 297/596] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:51.207760 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298e3f1-54c6-4a02-830b-c361e09a9e88 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:51.262120 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:51.301045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e3d28-6118-4f37-941b-f69d5f26cc28 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:51.325769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298e3f1-54c6-4a02-830b-c361e09a9e88 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:51.337387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389ffc7-e9ce-48c5-85ef-ab154e025b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:51.338106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389ffc7-e9ce-48c5-85ef-ab154e025b1a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:51.338389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389ffc7-e9ce-48c5-85ef-ab154e025b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:51.338659 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d389ffc7-e9ce-48c5-85ef-ab154e025b1a None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:51.339469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389ffc7-e9ce-48c5-85ef-ab154e025b1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:51.339708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389ffc7-e9ce-48c5-85ef-ab154e025b1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:51.341163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d389ffc7-e9ce-48c5-85ef-ab154e025b1a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:51.358640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 300/597] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:04:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:51.358640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389ffc7-e9ce-48c5-85ef-ab154e025b1a None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:51.370110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c45ed-a10e-4cfa-98c6-50532796d310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:51.371167 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c45ed-a10e-4cfa-98c6-50532796d310 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:51.371651 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c45ed-a10e-4cfa-98c6-50532796d310 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:51.372213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c45ed-a10e-4cfa-98c6-50532796d310 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:51.384966 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c45ed-a10e-4cfa-98c6-50532796d310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:51.386934 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c45ed-a10e-4cfa-98c6-50532796d310 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:51.402600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-928c45ed-a10e-4cfa-98c6-50532796d310 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:51.402600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:51.402600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:51.402600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:51.402600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:51.402600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:51.402600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:51.402600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:51.402600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:51.402600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:51.402600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:51.402600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:51.402600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:51.402600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:51.402600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:51.403514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:51.403514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:51.403514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:51.403514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:51.403514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:51.403514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:51.403514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:51.403514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:51.403514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:51.403514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:51.403514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:51.403514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:51.403514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:51.403514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:51.403514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:51.403514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:51.403514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:51.404496 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:51.404496 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:51.404496 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:51.404496 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:51.406728 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 298/598] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:51.407804 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c45ed-a10e-4cfa-98c6-50532796d310 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:51.438411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:51.469737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d389ffc7-e9ce-48c5-85ef-ab154e025b1a None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:51.497364 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:51.508389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbfab6-44c1-47dd-96e1-9468caf56e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:51.509050 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbfab6-44c1-47dd-96e1-9468caf56e3a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:51.509362 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbfab6-44c1-47dd-96e1-9468caf56e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:51.509611 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbfab6-44c1-47dd-96e1-9468caf56e3a None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:51.510286 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbfab6-44c1-47dd-96e1-9468caf56e3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:51.510584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbfab6-44c1-47dd-96e1-9468caf56e3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:51.511939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbfab6-44c1-47dd-96e1-9468caf56e3a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:51.521755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c45ed-a10e-4cfa-98c6-50532796d310 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:51.524166 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 301/599] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:51.528659 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbfab6-44c1-47dd-96e1-9468caf56e3a None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:51.561331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888f853-da03-40e7-8dc1-4099f4d8ddcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:51.562504 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888f853-da03-40e7-8dc1-4099f4d8ddcc None admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:51.563090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888f853-da03-40e7-8dc1-4099f4d8ddcc None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:51.563517 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888f853-da03-40e7-8dc1-4099f4d8ddcc None admin] PATH_INFO: `/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/14ff31d52ecf4c7d93a65bbfca7285c0/roles/977824a8c6a24c00af2f6dabd0e2a6f0` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:51.582667 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888f853-da03-40e7-8dc1-4099f4d8ddcc None admin] RBAC: Authorizing `identity:create_grant(project_id=74b8caff828a4f5289094fb2ae2278f7, user_id=14ff31d52ecf4c7d93a65bbfca7285c0, role_id=977824a8c6a24c00af2f6dabd0e2a6f0)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:51.584172 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888f853-da03-40e7-8dc1-4099f4d8ddcc None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:51.608723 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 299/600] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:04:51 2022] PUT /identity/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/14ff31d52ecf4c7d93a65bbfca7285c0/roles/977824a8c6a24c00af2f6dabd0e2a6f0 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:04:51.612021 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6eaca-c22a-4b78-893e-c77be463670e None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:51.613498 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6eaca-c22a-4b78-893e-c77be463670e None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:51.614038 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6eaca-c22a-4b78-893e-c77be463670e None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:51.616438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 300/601] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:51.617471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6eaca-c22a-4b78-893e-c77be463670e None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:51.644255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbfab6-44c1-47dd-96e1-9468caf56e3a None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:51.688431 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b85331-5785-4cfa-a4ad-818875933721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:51.689108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b85331-5785-4cfa-a4ad-818875933721 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:51.689404 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b85331-5785-4cfa-a4ad-818875933721 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:51.689671 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b85331-5785-4cfa-a4ad-818875933721 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:51.690434 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b85331-5785-4cfa-a4ad-818875933721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:51.690737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b85331-5785-4cfa-a4ad-818875933721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:51.692152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b85331-5785-4cfa-a4ad-818875933721 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:51.711071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 302/602] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:51.711071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2a6ab-2b24-4676-a70c-f5ae15a763ce None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:51.711071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2a6ab-2b24-4676-a70c-f5ae15a763ce None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:51.711071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2a6ab-2b24-4676-a70c-f5ae15a763ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:51.729807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-cec2a6ab-2b24-4676-a70c-f5ae15a763ce None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:51.731084 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-cec2a6ab-2b24-4676-a70c-f5ae15a763ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:51.752295 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:51.796010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:51.796224 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6eaca-c22a-4b78-893e-c77be463670e None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:51.829489 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 303/603] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:51.834794 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2a6ab-2b24-4676-a70c-f5ae15a763ce None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:51.855693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44079f-a280-413c-adb1-c863cb33ad0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:51.856343 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44079f-a280-413c-adb1-c863cb33ad0e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:51.856696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44079f-a280-413c-adb1-c863cb33ad0e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:51.856970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44079f-a280-413c-adb1-c863cb33ad0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:51.866256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44079f-a280-413c-adb1-c863cb33ad0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:51.867713 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c44079f-a280-413c-adb1-c863cb33ad0e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:51.883013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-4c44079f-a280-413c-adb1-c863cb33ad0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:51.883013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:51.883013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:51.883013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:51.883013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:51.883013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:51.883013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:51.883013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:51.883013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:51.883013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:51.883013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:51.883013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:51.883013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:04:51.883013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:04:51.883013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:04:51.883912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:04:51.883912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:51.883912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:51.883912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:51.883912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:51.883912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:51.883912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:51.883912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:51.883912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:51.883912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:51.883912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:51.883912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:51.883912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:51.883912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:51.883912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:51.883912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:51.883912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:51.885237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:04:51.885237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:04:51.885237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:04:51.885237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:04:51.885237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:04:51.885237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:04:51.885237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:51.885237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:51.885237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 301/604] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:51.885715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44079f-a280-413c-adb1-c863cb33ad0e None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:51.972368 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2a6ab-2b24-4676-a70c-f5ae15a763ce None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:52.013582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44079f-a280-413c-adb1-c863cb33ad0e None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:52.014765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b4e3f-4f97-4c6f-9ac9-6662fefb3c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:52.016501 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b4e3f-4f97-4c6f-9ac9-6662fefb3c88 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:52.016985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b4e3f-4f97-4c6f-9ac9-6662fefb3c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:52.017431 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b4e3f-4f97-4c6f-9ac9-6662fefb3c88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:52.025927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b4e3f-4f97-4c6f-9ac9-6662fefb3c88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:52.028236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b4e3f-4f97-4c6f-9ac9-6662fefb3c88 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:52.043302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-0e0b4e3f-4f97-4c6f-9ac9-6662fefb3c88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:52.043302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:52.043302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:52.043302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:52.043302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:52.043302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:52.043302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:52.043302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:52.043302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:52.043302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:52.043302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:52.043302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:52.043302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:04:52.043302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:04:52.043302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:04:52.044195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:04:52.044195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:52.044195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:52.044195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:52.044195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:52.044195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:52.044195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:52.044195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:52.044195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:52.044195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:52.044195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:52.044195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:52.044195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:52.044195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:52.044195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:52.044195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:52.044195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:52.045135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:04:52.045135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:04:52.045135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:04:52.045135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:04:52.045135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:04:52.045135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:04:52.045135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:52.045135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:52.047218 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 304/605] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:52.048304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b4e3f-4f97-4c6f-9ac9-6662fefb3c88 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:52.059199 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f12d3-debd-430b-8bd8-8e8d450c9e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:52.059908 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f12d3-debd-430b-8bd8-8e8d450c9e41 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:52.060187 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f12d3-debd-430b-8bd8-8e8d450c9e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:52.060547 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f12d3-debd-430b-8bd8-8e8d450c9e41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:52.063850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:52.074136 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02f12d3-debd-430b-8bd8-8e8d450c9e41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:52.079206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-d02f12d3-debd-430b-8bd8-8e8d450c9e41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:52.079206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:52.079206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:52.079206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:52.079206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:52.079206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:52.079206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:52.079206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:52.079206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:52.079206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:52.079206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:52.079206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:52.079206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:52.079206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:52.079206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:52.079206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:52.080148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:52.080148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:52.080148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:52.080148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:52.080148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:52.080148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:52.080148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:52.080148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:52.080148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:52.080148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:52.080148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:52.080148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:52.080148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:52.080148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:52.080148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:52.080148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:52.081553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:52.081553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:52.081553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:52.081553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:52.081553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:52.081553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:52.081553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:52.081553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:52.081553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:52.081553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:52.081553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:52.081553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:52.084946 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 302/606] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:52.085869 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f12d3-debd-430b-8bd8-8e8d450c9e41 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:52.097383 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:52.166694 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b4e3f-4f97-4c6f-9ac9-6662fefb3c88 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:52.202616 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77467d-0fd3-4aa7-b31e-3b9d76ba1177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:52.203371 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77467d-0fd3-4aa7-b31e-3b9d76ba1177 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:52.203762 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77467d-0fd3-4aa7-b31e-3b9d76ba1177 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:52.204125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77467d-0fd3-4aa7-b31e-3b9d76ba1177 None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:52.204913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f12d3-debd-430b-8bd8-8e8d450c9e41 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:52.205101 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77467d-0fd3-4aa7-b31e-3b9d76ba1177 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:52.205494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77467d-0fd3-4aa7-b31e-3b9d76ba1177 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:52.207543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77467d-0fd3-4aa7-b31e-3b9d76ba1177 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:52.219759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 305/607] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:04:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:52.220836 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77467d-0fd3-4aa7-b31e-3b9d76ba1177 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:52.244271 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00190bb-be23-4c08-8346-6dce853c3544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:52.245227 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00190bb-be23-4c08-8346-6dce853c3544 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:52.245652 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00190bb-be23-4c08-8346-6dce853c3544 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:52.246067 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00190bb-be23-4c08-8346-6dce853c3544 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:52.247144 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00190bb-be23-4c08-8346-6dce853c3544 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:52.247680 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00190bb-be23-4c08-8346-6dce853c3544 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:52.249536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00190bb-be23-4c08-8346-6dce853c3544 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:52.267845 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 303/608] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:04:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:52.270015 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00190bb-be23-4c08-8346-6dce853c3544 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:52.319853 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:52.325746 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77467d-0fd3-4aa7-b31e-3b9d76ba1177 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:52.336973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:52.362065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45828051-6769-4694-8de0-961ab94b27b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:52.363298 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45828051-6769-4694-8de0-961ab94b27b5 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:52.363609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45828051-6769-4694-8de0-961ab94b27b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:52.364183 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45828051-6769-4694-8de0-961ab94b27b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:52.364702 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45828051-6769-4694-8de0-961ab94b27b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:52.365003 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45828051-6769-4694-8de0-961ab94b27b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:52.366462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45828051-6769-4694-8de0-961ab94b27b5 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:52.380094 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 306/609] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:04:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:52.380094 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45828051-6769-4694-8de0-961ab94b27b5 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:52.394835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00190bb-be23-4c08-8346-6dce853c3544 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:52.432161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b0908-0b8d-40b5-a19a-afe97a6b02e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:52.433326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b0908-0b8d-40b5-a19a-afe97a6b02e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:52.433809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b0908-0b8d-40b5-a19a-afe97a6b02e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:52.434280 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b0908-0b8d-40b5-a19a-afe97a6b02e4 None admin] PATH_INFO: `/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/f4b8838129174fbb8cdd32bd5d4d6101/roles/977824a8c6a24c00af2f6dabd0e2a6f0` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:52.450774 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b0908-0b8d-40b5-a19a-afe97a6b02e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b8caff828a4f5289094fb2ae2278f7, user_id=f4b8838129174fbb8cdd32bd5d4d6101, role_id=977824a8c6a24c00af2f6dabd0e2a6f0)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:52.452778 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295b0908-0b8d-40b5-a19a-afe97a6b02e4 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:52.472865 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 304/610] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:04:52 2022] PUT /identity/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/f4b8838129174fbb8cdd32bd5d4d6101/roles/977824a8c6a24c00af2f6dabd0e2a6f0 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:04:52.474592 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b0908-0b8d-40b5-a19a-afe97a6b02e4 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:52.487853 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45828051-6769-4694-8de0-961ab94b27b5 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:52.532238 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c547df9-5e77-4e79-b848-8e045d4e7333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:52.533638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c547df9-5e77-4e79-b848-8e045d4e7333 None admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:52.534672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c547df9-5e77-4e79-b848-8e045d4e7333 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:52.535314 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c547df9-5e77-4e79-b848-8e045d4e7333 None admin] PATH_INFO: `/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/6c2da427acb24c8cb6d667c6347858c7/roles/977824a8c6a24c00af2f6dabd0e2a6f0` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:52.562559 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c547df9-5e77-4e79-b848-8e045d4e7333 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b8caff828a4f5289094fb2ae2278f7, user_id=6c2da427acb24c8cb6d667c6347858c7, role_id=977824a8c6a24c00af2f6dabd0e2a6f0)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:52.565569 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c547df9-5e77-4e79-b848-8e045d4e7333 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:52.584695 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 307/611] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:04:52 2022] PUT /identity/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/6c2da427acb24c8cb6d667c6347858c7/roles/977824a8c6a24c00af2f6dabd0e2a6f0 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:04:52.609233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b0908-0b8d-40b5-a19a-afe97a6b02e4 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:52.619060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:52.658905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a4114b-2308-44ca-9c8c-852d6d2f4321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:52.659742 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4114b-2308-44ca-9c8c-852d6d2f4321 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:52.660020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4114b-2308-44ca-9c8c-852d6d2f4321 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:52.660288 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4114b-2308-44ca-9c8c-852d6d2f4321 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:52.681179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4114b-2308-44ca-9c8c-852d6d2f4321 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:52.682812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a4114b-2308-44ca-9c8c-852d6d2f4321 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:52.704601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-a5a4114b-2308-44ca-9c8c-852d6d2f4321 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:52.704601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:52.704601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:52.704601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:52.704601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:52.704601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:52.704601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:52.704601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:52.704601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:52.704601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:52.704601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:52.704601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:52.704601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:52.704601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:52.704601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:52.705577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:52.705577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:52.705577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:52.705577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:52.705577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:52.705577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:52.705577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:52.705577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:52.705577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:52.705577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:52.705577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:52.705577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:52.705577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:52.705577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:52.705577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:52.705577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:52.705577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:52.707050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:52.707050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:52.707050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:52.707050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:52.712650 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 305/612] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:52.715458 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c547df9-5e77-4e79-b848-8e045d4e7333 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:52.739249 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:52.830169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b8569-9a78-41df-ae1b-9ba670c36523 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:52.830671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b8569-9a78-41df-ae1b-9ba670c36523 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:52.830671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b8569-9a78-41df-ae1b-9ba670c36523 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:52.832766 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 306/613] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:04:52.838103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89207a7e-fe0e-41e2-aa09-f3aff4aefaf2 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:52.838388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89207a7e-fe0e-41e2-aa09-f3aff4aefaf2 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:52.838660 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89207a7e-fe0e-41e2-aa09-f3aff4aefaf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:52.851229 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c547df9-5e77-4e79-b848-8e045d4e7333 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:52.866943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-89207a7e-fe0e-41e2-aa09-f3aff4aefaf2 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:52.868162 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-89207a7e-fe0e-41e2-aa09-f3aff4aefaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:52.901930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ff6f6-304d-47e8-9188-7b91b6419ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:52.903049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ff6f6-304d-47e8-9188-7b91b6419ada None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:52.903683 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ff6f6-304d-47e8-9188-7b91b6419ada None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:52.904007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478ff6f6-304d-47e8-9188-7b91b6419ada None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:52.904730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ff6f6-304d-47e8-9188-7b91b6419ada None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:52.905049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ff6f6-304d-47e8-9188-7b91b6419ada None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:52.906642 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478ff6f6-304d-47e8-9188-7b91b6419ada None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:52.920084 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 308/614] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:52.927747 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ff6f6-304d-47e8-9188-7b91b6419ada None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:52.945971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 307/615] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 04 02:04:52.969633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89207a7e-fe0e-41e2-aa09-f3aff4aefaf2 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:52.998489 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:53.071015 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:53.075584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478ff6f6-304d-47e8-9188-7b91b6419ada None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:53.084954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89207a7e-fe0e-41e2-aa09-f3aff4aefaf2 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:53.122798 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626544a-b845-4ce6-9dcf-d37e1d98c186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:53.123448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626544a-b845-4ce6-9dcf-d37e1d98c186 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:53.123789 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626544a-b845-4ce6-9dcf-d37e1d98c186 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:53.123993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626544a-b845-4ce6-9dcf-d37e1d98c186 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:53.146608 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2626544a-b845-4ce6-9dcf-d37e1d98c186 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:53.152344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-2626544a-b845-4ce6-9dcf-d37e1d98c186 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:53.152344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:53.152344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:53.152344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:53.152344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:53.152344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:53.152344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:53.152344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:53.152344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:53.152344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:53.152344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:53.152344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:53.152344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:53.152344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:53.152344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:53.152344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:53.153325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:53.153325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:53.153325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:53.153325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:53.153325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:53.153325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:53.153325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:53.153325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:53.153325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:53.153325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:53.153325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:53.153325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:53.153325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:53.153325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:53.153325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:53.153325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:53.154159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:53.154159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:53.154159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:53.154159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:53.154159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:53.154159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:53.154159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:53.154159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:53.154159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:53.154159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:53.154159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:53.154159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:53.155950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b97b7-172a-49aa-a12b-01a67c9f49ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:53.156861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b97b7-172a-49aa-a12b-01a67c9f49ae None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:53.157338 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b97b7-172a-49aa-a12b-01a67c9f49ae None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:53.157837 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b97b7-172a-49aa-a12b-01a67c9f49ae None admin] PATH_INFO: `/v3/users/swift` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:53.167293 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b97b7-172a-49aa-a12b-01a67c9f49ae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:53.167629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 308/616] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:53.173675 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b97b7-172a-49aa-a12b-01a67c9f49ae None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:53.176609 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626544a-b845-4ce6-9dcf-d37e1d98c186 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:53.199376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-9c8b97b7-172a-49aa-a12b-01a67c9f49ae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 02:04:53.199376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:53.199376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:53.199376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:53.199376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:53.199376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:53.199376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:53.199376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:53.199376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:53.199376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:53.199376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:53.199376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:53.199376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:04:53.199376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:04:53.199376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:04:53.199376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:53.200394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:53.201356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:53.201356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:53.201356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:04:53.201356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:04:53.201356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:04:53.201356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:04:53.201356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:04:53.201356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:04:53.201356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 02:04:53.201356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:53.206670 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 309/617] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:04:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:04:53.211311 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b97b7-172a-49aa-a12b-01a67c9f49ae None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:53.342453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2626544a-b845-4ce6-9dcf-d37e1d98c186 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:53.380702 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef40797-10f5-4818-b0e5-a929b88d602a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:53.381316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef40797-10f5-4818-b0e5-a929b88d602a None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:53.381592 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef40797-10f5-4818-b0e5-a929b88d602a None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:53.381861 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef40797-10f5-4818-b0e5-a929b88d602a None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:53.382579 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef40797-10f5-4818-b0e5-a929b88d602a None admin] RBAC: Adding query filter params (name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:53.382885 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef40797-10f5-4818-b0e5-a929b88d602a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:53.384280 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef40797-10f5-4818-b0e5-a929b88d602a None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:53.397649 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b97b7-172a-49aa-a12b-01a67c9f49ae None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:53.399673 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 309/618] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:04:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:53.405334 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef40797-10f5-4818-b0e5-a929b88d602a None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:53.435259 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:53.446974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:53.466573 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081dfc7-da06-4b74-8e29-22fd7c315c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:53.467193 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081dfc7-da06-4b74-8e29-22fd7c315c00 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:53.467471 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081dfc7-da06-4b74-8e29-22fd7c315c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:53.467804 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081dfc7-da06-4b74-8e29-22fd7c315c00 None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:53.468609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081dfc7-da06-4b74-8e29-22fd7c315c00 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:53.468932 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081dfc7-da06-4b74-8e29-22fd7c315c00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:53.470825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7081dfc7-da06-4b74-8e29-22fd7c315c00 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:53.493033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 310/619] 10.222.0.45 () {56 vars in 1135 bytes} [Thu Aug 4 02:04:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:53.498304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081dfc7-da06-4b74-8e29-22fd7c315c00 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:53.601294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081dfc7-da06-4b74-8e29-22fd7c315c00 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:53.652782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b545d2-7ff3-4948-8529-6cf84fec09ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:53.652782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b545d2-7ff3-4948-8529-6cf84fec09ff None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:53.652782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b545d2-7ff3-4948-8529-6cf84fec09ff None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:53.652782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b545d2-7ff3-4948-8529-6cf84fec09ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:53.668985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b545d2-7ff3-4948-8529-6cf84fec09ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:53.671030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b545d2-7ff3-4948-8529-6cf84fec09ff None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:53.687796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-47b545d2-7ff3-4948-8529-6cf84fec09ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:53.687796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:53.687796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:53.687796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:53.687796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:53.687796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:53.687796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:53.687796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:53.687796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:53.687796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:53.687796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:53.687796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:53.687796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:53.687796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:53.687796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:53.688968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:53.688968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:53.688968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:53.688968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:53.688968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:53.688968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:53.688968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:53.688968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:53.688968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:53.688968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:53.688968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:53.688968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:53.688968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:53.688968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:53.688968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:53.688968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:53.688968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:53.689845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:53.689845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:53.689845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:53.689845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:53.689845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 311/620] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:53.695974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b545d2-7ff3-4948-8529-6cf84fec09ff None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:53.702589 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef40797-10f5-4818-b0e5-a929b88d602a None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:53.744428 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:53.775807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b6e9e-bfef-4873-9f9f-198a0e2a76e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:53.777838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b6e9e-bfef-4873-9f9f-198a0e2a76e2 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:53.778375 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b6e9e-bfef-4873-9f9f-198a0e2a76e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:53.778876 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b6e9e-bfef-4873-9f9f-198a0e2a76e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:53.804130 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b6e9e-bfef-4873-9f9f-198a0e2a76e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:53.807179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3b6e9e-bfef-4873-9f9f-198a0e2a76e2 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:53.837961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-2d3b6e9e-bfef-4873-9f9f-198a0e2a76e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:53.837961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:53.837961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:53.837961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:53.837961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:53.837961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:53.837961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:53.837961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:53.837961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:53.837961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:53.837961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:53.837961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:53.837961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:53.837961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:53.837961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:53.839332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:53.839332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:53.839332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:53.839332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:53.839332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:53.839332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:53.839332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:53.839332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:53.839332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:53.839332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:53.839332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:53.839332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:53.839332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:53.839332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:53.839332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:53.839332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:53.839332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:53.840262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:53.840262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:53.840262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:53.840262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:53.848583 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 310/621] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 444 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:04:53.853785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b6e9e-bfef-4873-9f9f-198a0e2a76e2 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:53.867610 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b545d2-7ff3-4948-8529-6cf84fec09ff None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:53.903865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf14b79-6072-4739-be43-c6b9d2619d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:53.904094 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf14b79-6072-4739-be43-c6b9d2619d3b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:53.904319 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf14b79-6072-4739-be43-c6b9d2619d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:53.904534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf14b79-6072-4739-be43-c6b9d2619d3b None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:53.905268 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf14b79-6072-4739-be43-c6b9d2619d3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:53.905570 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf14b79-6072-4739-be43-c6b9d2619d3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:53.906935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf14b79-6072-4739-be43-c6b9d2619d3b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:53.920705 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 312/622] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:53.921981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac2bb0-c0ef-4803-85f5-2feb1a37624c None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:53.922265 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac2bb0-c0ef-4803-85f5-2feb1a37624c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:53.922541 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac2bb0-c0ef-4803-85f5-2feb1a37624c None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:53.924018 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 313/623] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:53.927733 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac2bb0-c0ef-4803-85f5-2feb1a37624c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:53.939110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:53.991679 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3b6e9e-bfef-4873-9f9f-198a0e2a76e2 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:54.030071 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43c4f5-c022-47dd-bcac-938914421bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:54.030696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43c4f5-c022-47dd-bcac-938914421bfb None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:54.030974 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43c4f5-c022-47dd-bcac-938914421bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:54.031245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43c4f5-c022-47dd-bcac-938914421bfb None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:54.032098 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43c4f5-c022-47dd-bcac-938914421bfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:54.032404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43c4f5-c022-47dd-bcac-938914421bfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:54.033841 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43c4f5-c022-47dd-bcac-938914421bfb None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:54.041831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:54.047389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ac2bb0-c0ef-4803-85f5-2feb1a37624c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:54.048563 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 311/624] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:54.051355 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f94eff-72c1-403a-bf2e-9c80c7f3b9e8 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:54.051638 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f94eff-72c1-403a-bf2e-9c80c7f3b9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:54.051907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f94eff-72c1-403a-bf2e-9c80c7f3b9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:54.071049 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-b9f94eff-72c1-403a-bf2e-9c80c7f3b9e8 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:54.072228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-b9f94eff-72c1-403a-bf2e-9c80c7f3b9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:54.088274 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9038c2-ff70-48f3-bdb3-116f8037986e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:54.089587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9038c2-ff70-48f3-bdb3-116f8037986e None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:54.090102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9038c2-ff70-48f3-bdb3-116f8037986e None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:54.090753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9038c2-ff70-48f3-bdb3-116f8037986e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:54.100902 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9038c2-ff70-48f3-bdb3-116f8037986e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:54.103127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9038c2-ff70-48f3-bdb3-116f8037986e None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:54.118354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-fa9038c2-ff70-48f3-bdb3-116f8037986e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:54.118354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:54.118354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:54.118354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:54.118354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:54.118354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:54.118354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:54.118354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:54.118354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:54.118354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:54.118354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:54.118354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:54.118354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:04:54.118354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:04:54.118354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:04:54.119264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:04:54.119264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:54.119264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:54.119264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:54.119264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:54.119264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:54.119264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:54.119264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:54.119264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:54.119264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:54.119264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:54.119264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:54.119264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:54.119264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:54.119264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:54.119264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:54.119264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:54.120164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:04:54.120164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:04:54.120164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:04:54.120164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:04:54.120164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:04:54.120164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:04:54.120164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:54.120164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:54.123014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 314/625] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:54.124303 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9038c2-ff70-48f3-bdb3-116f8037986e None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:54.147811 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 312/626] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:54.148787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f94eff-72c1-403a-bf2e-9c80c7f3b9e8 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:54.221015 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:54.241016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9038c2-ff70-48f3-bdb3-116f8037986e None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:54.274303 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f94eff-72c1-403a-bf2e-9c80c7f3b9e8 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:54.283622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b1455-35f9-42fd-bf7c-ada353efcc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:54.285145 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b1455-35f9-42fd-bf7c-ada353efcc80 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:54.285521 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b1455-35f9-42fd-bf7c-ada353efcc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:54.285886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b1455-35f9-42fd-bf7c-ada353efcc80 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:54.296056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b1455-35f9-42fd-bf7c-ada353efcc80 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:54.298069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b1455-35f9-42fd-bf7c-ada353efcc80 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:54.313842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-2c3b1455-35f9-42fd-bf7c-ada353efcc80 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:04:54.313842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:54.313842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:54.313842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:54.313842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:54.313842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:54.313842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:54.313842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:54.313842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:54.313842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:54.313842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:54.313842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:54.313842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:04:54.313842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:04:54.313842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:04:54.313842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:54.314931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:54.316475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:54.316475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:54.316475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:04:54.316475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:04:54.316475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:04:54.316475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:04:54.316475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:04:54.316475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:04:54.316475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:04:54.316475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:54.320705 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 315/627] 10.222.0.45 () {56 vars in 1093 bytes} [Thu Aug 4 02:04:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:54.321613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556aa4b-1342-432d-aebe-b8d610fb4174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:54.322723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b1455-35f9-42fd-bf7c-ada353efcc80 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:54.323730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556aa4b-1342-432d-aebe-b8d610fb4174 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:54.324008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556aa4b-1342-432d-aebe-b8d610fb4174 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:54.324278 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556aa4b-1342-432d-aebe-b8d610fb4174 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:54.325070 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556aa4b-1342-432d-aebe-b8d610fb4174 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:54.325380 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556aa4b-1342-432d-aebe-b8d610fb4174 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:54.326821 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b556aa4b-1342-432d-aebe-b8d610fb4174 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:54.342246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 313/628] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:04:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:54.343108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556aa4b-1342-432d-aebe-b8d610fb4174 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:54.350060 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:54.465267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b1455-35f9-42fd-bf7c-ada353efcc80 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:54.477158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556aa4b-1342-432d-aebe-b8d610fb4174 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:54.494368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:54.513568 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76ffc2-3aa1-4f24-99a9-a6dd349f7dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:54.514830 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76ffc2-3aa1-4f24-99a9-a6dd349f7dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:54.515715 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76ffc2-3aa1-4f24-99a9-a6dd349f7dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:54.516321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76ffc2-3aa1-4f24-99a9-a6dd349f7dd6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:54.521587 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87286281-6b01-4672-8474-11666d84a27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:54.522375 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87286281-6b01-4672-8474-11666d84a27b None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:54.522660 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87286281-6b01-4672-8474-11666d84a27b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:54.522932 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87286281-6b01-4672-8474-11666d84a27b None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:54.523739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87286281-6b01-4672-8474-11666d84a27b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:54.524206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87286281-6b01-4672-8474-11666d84a27b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:54.525848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87286281-6b01-4672-8474-11666d84a27b None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:54.531452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76ffc2-3aa1-4f24-99a9-a6dd349f7dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:54.539399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-ed76ffc2-3aa1-4f24-99a9-a6dd349f7dd6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:54.539399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:54.539399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:54.539399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:54.539399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:54.539399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:54.539399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:54.539399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:54.539399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:54.539399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:54.539399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:54.539399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:54.539399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:54.539399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:54.539399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:54.539399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:54.540375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:54.540375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:54.540375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:54.540375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:54.540375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:54.540375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:54.540375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:54.540375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:54.540375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:54.540375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:54.540375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:54.540375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:54.540375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:54.540375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:54.540375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:54.540375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:54.541235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:54.541235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:54.541235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:54.541235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:54.541235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:54.541235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:54.541235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:54.541235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:54.541235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:54.541235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:54.541235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:54.541235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:54.544810 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 316/629] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:54.545976 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76ffc2-3aa1-4f24-99a9-a6dd349f7dd6 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:54.550991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 314/630] 10.222.0.45 () {56 vars in 1133 bytes} [Thu Aug 4 02:04:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:54.550991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87286281-6b01-4672-8474-11666d84a27b None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:54.634900 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:54.660342 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76ffc2-3aa1-4f24-99a9-a6dd349f7dd6 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:54.665954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87286281-6b01-4672-8474-11666d84a27b None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:54.696251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbcf0e-f5b8-4464-873f-e22a02b00821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:54.696989 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbcf0e-f5b8-4464-873f-e22a02b00821 None admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:54.697325 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbcf0e-f5b8-4464-873f-e22a02b00821 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:54.697614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbcf0e-f5b8-4464-873f-e22a02b00821 None admin] PATH_INFO: `/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/155a7276efab426cb34175683c475f33/roles/977824a8c6a24c00af2f6dabd0e2a6f0` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:54.708393 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bd724-d124-467e-b2db-583ca1fd78a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:54.709506 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bd724-d124-467e-b2db-583ca1fd78a5 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:54.709984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bd724-d124-467e-b2db-583ca1fd78a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:54.710447 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95bd724-d124-467e-b2db-583ca1fd78a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:54.711469 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95bd724-d124-467e-b2db-583ca1fd78a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:54.711991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95bd724-d124-467e-b2db-583ca1fd78a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:54.713661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95bd724-d124-467e-b2db-583ca1fd78a5 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:54.715377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbcf0e-f5b8-4464-873f-e22a02b00821 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b8caff828a4f5289094fb2ae2278f7, user_id=155a7276efab426cb34175683c475f33, role_id=977824a8c6a24c00af2f6dabd0e2a6f0)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:54.716806 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbcf0e-f5b8-4464-873f-e22a02b00821 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:54.724467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 315/631] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:04:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:04:54.725977 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bd724-d124-467e-b2db-583ca1fd78a5 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:54.731217 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 317/632] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:04:54 2022] PUT /identity/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/155a7276efab426cb34175683c475f33/roles/977824a8c6a24c00af2f6dabd0e2a6f0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:04:54.732160 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbcf0e-f5b8-4464-873f-e22a02b00821 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:54.763174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:54.848155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95bd724-d124-467e-b2db-583ca1fd78a5 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:54.853773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbcf0e-f5b8-4464-873f-e22a02b00821 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:54.883311 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:54.893063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa61a1b-5e9f-408f-9ce0-584bade0b117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:54.894691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee211af-b618-4833-89a5-261391b9e834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:54.895283 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa61a1b-5e9f-408f-9ce0-584bade0b117 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:54.896110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee211af-b618-4833-89a5-261391b9e834 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:54.896230 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa61a1b-5e9f-408f-9ce0-584bade0b117 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:54.896726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee211af-b618-4833-89a5-261391b9e834 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:54.896889 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa61a1b-5e9f-408f-9ce0-584bade0b117 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:54.897312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee211af-b618-4833-89a5-261391b9e834 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:54.910554 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa61a1b-5e9f-408f-9ce0-584bade0b117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:54.912279 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa61a1b-5e9f-408f-9ce0-584bade0b117 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:54.928323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-6fa61a1b-5e9f-408f-9ce0-584bade0b117 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:54.928323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:54.928323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:54.928323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:54.928323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:54.928323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:54.928323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:54.928323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:54.928323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:54.928323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:54.928323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:54.928323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:54.928323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:54.928323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:54.928323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:54.929480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:54.929480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:54.929480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:54.929480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:54.929480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:54.929480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:54.929480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:54.929480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:54.929480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:54.929480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:54.929480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:54.929480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:54.929480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:54.929480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:54.929480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:54.929480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:54.929480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:54.930392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:54.930392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:54.930392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:54.930392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:54.931093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 318/633] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:54.938757 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64c134-1b92-4412-b402-0b8604307756 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:54.939169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64c134-1b92-4412-b402-0b8604307756 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:54.939440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64c134-1b92-4412-b402-0b8604307756 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:54.941804 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 319/634] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:54.942602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64c134-1b92-4412-b402-0b8604307756 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:54.995217 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee211af-b618-4833-89a5-261391b9e834 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:54.996652 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee211af-b618-4833-89a5-261391b9e834 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:55.032879 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:55.070177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64c134-1b92-4412-b402-0b8604307756 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:55.106387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00535c3-a36e-4a04-b053-e62a9c040a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:55.107009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00535c3-a36e-4a04-b053-e62a9c040a55 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:55.107285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00535c3-a36e-4a04-b053-e62a9c040a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:55.107560 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00535c3-a36e-4a04-b053-e62a9c040a55 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:55.108251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00535c3-a36e-4a04-b053-e62a9c040a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:55.108554 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00535c3-a36e-4a04-b053-e62a9c040a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:55.109948 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00535c3-a36e-4a04-b053-e62a9c040a55 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:55.111349 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 316/635] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:55.112660 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b63d0-05f9-4220-aff2-8b41dae7a92d None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:55.112950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b63d0-05f9-4220-aff2-8b41dae7a92d None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:55.113230 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b63d0-05f9-4220-aff2-8b41dae7a92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:55.134824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 320/636] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:55.134824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1d2bc-2e87-4188-a497-e582efe24574 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:55.134824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1d2bc-2e87-4188-a497-e582efe24574 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:55.134824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1d2bc-2e87-4188-a497-e582efe24574 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:55.134824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 321/637] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:55.134824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1d2bc-2e87-4188-a497-e582efe24574 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:55.137482 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-bd9b63d0-05f9-4220-aff2-8b41dae7a92d None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:55.138719 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-bd9b63d0-05f9-4220-aff2-8b41dae7a92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:55.186222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:55.214508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 317/638] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:55.215357 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b63d0-05f9-4220-aff2-8b41dae7a92d None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:55.261806 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1d2bc-2e87-4188-a497-e582efe24574 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:55.284838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:55.300799 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a8378-6fd5-4583-bcc5-266c11f87e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:55.301445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a8378-6fd5-4583-bcc5-266c11f87e87 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:55.301723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a8378-6fd5-4583-bcc5-266c11f87e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:55.301994 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a8378-6fd5-4583-bcc5-266c11f87e87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:55.313581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a8378-6fd5-4583-bcc5-266c11f87e87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:55.315037 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a8378-6fd5-4583-bcc5-266c11f87e87 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:55.328230 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b63d0-05f9-4220-aff2-8b41dae7a92d None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:55.333879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-5f5a8378-6fd5-4583-bcc5-266c11f87e87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:55.333879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:55.333879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:55.333879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:55.333879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:55.333879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:55.333879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:55.333879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:55.333879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:55.333879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:55.333879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:55.333879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:55.333879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:55.333879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:55.333879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:55.334758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:55.334758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:55.334758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:55.334758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:55.334758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:55.334758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:55.334758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:55.334758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:55.334758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:55.334758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:55.334758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:55.334758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:55.334758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:55.334758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:55.334758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:55.334758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:55.334758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:55.340904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:55.340904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:55.340904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:55.340904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:55.340904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 322/639] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:55.340904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f88ee-b8b0-45a0-b7d6-f9114ca9875a None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:55.340904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f88ee-b8b0-45a0-b7d6-f9114ca9875a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:55.340904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f88ee-b8b0-45a0-b7d6-f9114ca9875a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:55.357911 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c84f88ee-b8b0-45a0-b7d6-f9114ca9875a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:55.359100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c84f88ee-b8b0-45a0-b7d6-f9114ca9875a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:55.365520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc94c3d-fccc-407a-b1c0-42ec514301c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:55.366162 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc94c3d-fccc-407a-b1c0-42ec514301c3 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:55.366438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc94c3d-fccc-407a-b1c0-42ec514301c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:55.366705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc94c3d-fccc-407a-b1c0-42ec514301c3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:55.374378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc94c3d-fccc-407a-b1c0-42ec514301c3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:55.375823 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc94c3d-fccc-407a-b1c0-42ec514301c3 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:55.388987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-ccc94c3d-fccc-407a-b1c0-42ec514301c3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 04 02:04:55.388987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:55.388987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:55.388987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:55.388987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:55.388987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:55.388987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:55.388987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:55.388987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:55.388987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:55.388987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:55.388987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:55.388987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:04:55.388987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:04:55.388987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:04:55.388987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:55.390010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:55.390961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:55.390961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:55.390961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:04:55.390961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:04:55.390961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:04:55.390961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:04:55.390961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:04:55.390961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:04:55.390961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 04 02:04:55.390961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:55.390961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 318/640] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:55.390961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc94c3d-fccc-407a-b1c0-42ec514301c3 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:55.444772 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 323/641] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:55.445970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f88ee-b8b0-45a0-b7d6-f9114ca9875a None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:55.475083 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:55.496910 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc94c3d-fccc-407a-b1c0-42ec514301c3 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:55.522547 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:55.537464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de325b-f0e2-4a61-86e8-f3c3bd7618b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:55.538336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de325b-f0e2-4a61-86e8-f3c3bd7618b3 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:55.538602 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de325b-f0e2-4a61-86e8-f3c3bd7618b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:55.538888 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de325b-f0e2-4a61-86e8-f3c3bd7618b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:55.549701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de325b-f0e2-4a61-86e8-f3c3bd7618b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:55.555041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-18de325b-f0e2-4a61-86e8-f3c3bd7618b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:55.555041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:55.555041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:55.555041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:55.555041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:55.555041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:55.555041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:55.555041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:55.555041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:55.555041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:55.555041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:55.555041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:55.555041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:55.555041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:55.555041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:55.555041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:55.556081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:55.556081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:55.556081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:55.556081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:55.556081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:55.556081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:55.556081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:55.556081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:55.556081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:55.556081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:55.556081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:55.556081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:55.556081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:55.556081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:55.556081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:55.556081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:55.556960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:55.556960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:55.556960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:55.556960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:55.556960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:55.556960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:55.556960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:55.556960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:55.556960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:55.556960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:55.556960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:55.556960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:55.556960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 319/642] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:55.557689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de325b-f0e2-4a61-86e8-f3c3bd7618b3 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:55.564829 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f88ee-b8b0-45a0-b7d6-f9114ca9875a None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:55.606547 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d5f80-b165-45ea-bf3d-9838a100e761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:55.607166 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d5f80-b165-45ea-bf3d-9838a100e761 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:55.607447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d5f80-b165-45ea-bf3d-9838a100e761 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:55.607722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d5f80-b165-45ea-bf3d-9838a100e761 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:55.608414 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d5f80-b165-45ea-bf3d-9838a100e761 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:55.608771 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d5f80-b165-45ea-bf3d-9838a100e761 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:55.610154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d5f80-b165-45ea-bf3d-9838a100e761 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:55.622343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 324/643] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:55.623048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d5f80-b165-45ea-bf3d-9838a100e761 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:55.683543 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de325b-f0e2-4a61-86e8-f3c3bd7618b3 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:55.718980 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c9d0e-acee-4587-8208-3f1796cd83ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:55.719628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c9d0e-acee-4587-8208-3f1796cd83ec None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:55.719879 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c9d0e-acee-4587-8208-3f1796cd83ec None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:55.720142 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c9d0e-acee-4587-8208-3f1796cd83ec None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:55.720963 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c9d0e-acee-4587-8208-3f1796cd83ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:55.721268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c9d0e-acee-4587-8208-3f1796cd83ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:55.722725 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99c9d0e-acee-4587-8208-3f1796cd83ec None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:55.723676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:55.743499 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d5f80-b165-45ea-bf3d-9838a100e761 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:55.744019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 320/644] 10.222.0.45 () {56 vars in 1137 bytes} [Thu Aug 4 02:04:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:55.744880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c9d0e-acee-4587-8208-3f1796cd83ec None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:55.782030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7924371-1d47-41be-83be-97f1fcdad359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:55.782690 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7924371-1d47-41be-83be-97f1fcdad359 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:55.782971 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7924371-1d47-41be-83be-97f1fcdad359 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:55.783249 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7924371-1d47-41be-83be-97f1fcdad359 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:55.791076 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:55.793731 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7924371-1d47-41be-83be-97f1fcdad359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:55.798966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-b7924371-1d47-41be-83be-97f1fcdad359 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:55.798966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:55.798966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:55.798966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:55.798966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:55.798966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:55.798966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:55.798966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:55.798966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:55.798966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:55.798966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:55.798966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:55.798966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:55.798966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:55.798966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:55.798966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:55.800429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:55.800429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:55.800429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:55.800429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:55.800429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:55.800429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:55.800429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:55.800429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:55.800429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:55.800429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:55.800429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:55.800429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:55.800429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:55.800429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:55.800429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:55.800429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:55.801387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:55.801387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:55.801387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:55.801387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:55.801387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:55.801387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:55.801387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:55.801387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:55.801387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:55.801387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:55.801387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:55.801387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:55.801387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 325/645] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:55.802185 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7924371-1d47-41be-83be-97f1fcdad359 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:55.861778 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99c9d0e-acee-4587-8208-3f1796cd83ec None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:55.909955 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806cc87-9524-422a-9ef3-7b15e022350e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:55.910689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806cc87-9524-422a-9ef3-7b15e022350e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:55.910906 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806cc87-9524-422a-9ef3-7b15e022350e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:55.911183 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806cc87-9524-422a-9ef3-7b15e022350e None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:55.912041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806cc87-9524-422a-9ef3-7b15e022350e None admin] RBAC: Adding query filter params (name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:55.912263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806cc87-9524-422a-9ef3-7b15e022350e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:55.913740 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806cc87-9524-422a-9ef3-7b15e022350e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:55.915165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7924371-1d47-41be-83be-97f1fcdad359 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:55.922320 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 321/646] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:04:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:55.922983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806cc87-9524-422a-9ef3-7b15e022350e None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:55.956036 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ae285-f8cb-4020-9f34-93373834960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:55.956864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ae285-f8cb-4020-9f34-93373834960d None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:55.957219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ae285-f8cb-4020-9f34-93373834960d None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:55.957551 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6ae285-f8cb-4020-9f34-93373834960d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:55.966075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ae285-f8cb-4020-9f34-93373834960d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:55.967526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6ae285-f8cb-4020-9f34-93373834960d None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:55.980062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-2f6ae285-f8cb-4020-9f34-93373834960d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:55.980062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:55.980062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:55.980062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:55.980062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:55.980062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:55.980062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:55.980062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:55.980062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:55.980062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:55.980062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:55.980062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:55.980062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:04:55.980062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:04:55.980062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:04:55.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:04:55.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:55.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:55.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:55.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:55.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:55.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:55.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:55.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:55.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:55.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:55.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:55.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:55.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:55.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:55.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:55.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:55.982358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:04:55.982358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:04:55.982358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:04:55.982358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:04:55.982358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:04:55.982358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:04:55.982358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:55.982358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:55.982358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 326/647] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:55.982358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ae285-f8cb-4020-9f34-93373834960d None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:55.985241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:56.041966 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806cc87-9524-422a-9ef3-7b15e022350e None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:56.044273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:56.081161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef1954-2ac3-4f1b-8caa-1fc41f5508c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:56.082443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef1954-2ac3-4f1b-8caa-1fc41f5508c4 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:56.082965 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef1954-2ac3-4f1b-8caa-1fc41f5508c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:56.083289 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef1954-2ac3-4f1b-8caa-1fc41f5508c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:56.092971 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6ae285-f8cb-4020-9f34-93373834960d None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:56.130106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b6b35-43cd-4de8-bcf1-4f516a28b59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:56.130723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b6b35-43cd-4de8-bcf1-4f516a28b59b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:56.131002 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b6b35-43cd-4de8-bcf1-4f516a28b59b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:56.131277 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375b6b35-43cd-4de8-bcf1-4f516a28b59b None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:56.132008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b6b35-43cd-4de8-bcf1-4f516a28b59b None admin] RBAC: Adding query filter params (name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:56.132292 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b6b35-43cd-4de8-bcf1-4f516a28b59b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:56.133695 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375b6b35-43cd-4de8-bcf1-4f516a28b59b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:56.142476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 327/648] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:04:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:56.143263 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b6b35-43cd-4de8-bcf1-4f516a28b59b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:56.161735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef1954-2ac3-4f1b-8caa-1fc41f5508c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:56.163397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ef1954-2ac3-4f1b-8caa-1fc41f5508c4 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:56.205271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:56.245834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375b6b35-43cd-4de8-bcf1-4f516a28b59b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:56.267918 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 322/649] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:56.269107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef1954-2ac3-4f1b-8caa-1fc41f5508c4 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:56.273267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:56.287001 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3210c0d-b65d-4348-8243-92d8ebeaa281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:56.287646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3210c0d-b65d-4348-8243-92d8ebeaa281 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:56.287927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3210c0d-b65d-4348-8243-92d8ebeaa281 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:56.288197 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3210c0d-b65d-4348-8243-92d8ebeaa281 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:56.298998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3210c0d-b65d-4348-8243-92d8ebeaa281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:56.301041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3210c0d-b65d-4348-8243-92d8ebeaa281 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:56.317447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-a3210c0d-b65d-4348-8243-92d8ebeaa281 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:56.317447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:56.317447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:56.317447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:56.317447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:56.317447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:56.317447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:56.317447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:56.317447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:56.317447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:56.317447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:56.317447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:56.317447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:56.317447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:56.317447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:56.318411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:56.318411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:56.318411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:56.318411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:56.318411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:56.318411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:56.318411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:56.318411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:56.318411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:56.318411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:56.318411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:56.318411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:56.318411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:56.318411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:56.318411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:56.318411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:56.318411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:56.319312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:56.319312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:56.319312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:56.319312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:56.323335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 328/650] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:56.324263 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3210c0d-b65d-4348-8243-92d8ebeaa281 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:56.379789 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef1954-2ac3-4f1b-8caa-1fc41f5508c4 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:56.415483 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69e868-8d59-4209-ba5f-e3415f9ca949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:56.416247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69e868-8d59-4209-ba5f-e3415f9ca949 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:56.416635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69e868-8d59-4209-ba5f-e3415f9ca949 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:56.417432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69e868-8d59-4209-ba5f-e3415f9ca949 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:56.418278 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69e868-8d59-4209-ba5f-e3415f9ca949 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:56.418683 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69e868-8d59-4209-ba5f-e3415f9ca949 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:56.420226 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69e868-8d59-4209-ba5f-e3415f9ca949 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:56.429040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3210c0d-b65d-4348-8243-92d8ebeaa281 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:56.431763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 323/651] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:04:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:56.432566 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69e868-8d59-4209-ba5f-e3415f9ca949 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:56.451792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:56.472101 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92dcf31-0bd7-4e00-aa45-b66d293a22eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:56.472860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92dcf31-0bd7-4e00-aa45-b66d293a22eb None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:56.473212 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92dcf31-0bd7-4e00-aa45-b66d293a22eb None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:56.473555 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92dcf31-0bd7-4e00-aa45-b66d293a22eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:56.484587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92dcf31-0bd7-4e00-aa45-b66d293a22eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:56.486120 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92dcf31-0bd7-4e00-aa45-b66d293a22eb None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:56.498537 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:56.504231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-c92dcf31-0bd7-4e00-aa45-b66d293a22eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:56.504231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:56.504231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:56.504231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:56.504231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:56.504231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:56.504231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:56.504231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:56.504231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:56.504231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:56.504231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:56.504231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:56.504231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:56.504231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:56.504231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:56.505337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:56.505337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:56.505337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:56.505337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:56.505337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:56.505337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:56.505337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:56.505337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:56.505337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:56.505337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:56.505337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:56.505337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:56.505337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:56.505337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:56.505337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:56.505337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:56.505337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:56.506390 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:56.506390 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:56.506390 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:56.506390 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:56.510530 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 329/652] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:56.511218 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92dcf31-0bd7-4e00-aa45-b66d293a22eb None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:56.545200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69e868-8d59-4209-ba5f-e3415f9ca949 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:56.581758 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2818d5-2a22-450b-ab8d-cb894aa8b794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:56.582766 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2818d5-2a22-450b-ab8d-cb894aa8b794 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:56.583206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2818d5-2a22-450b-ab8d-cb894aa8b794 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:56.584110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2818d5-2a22-450b-ab8d-cb894aa8b794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:56.596895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2818d5-2a22-450b-ab8d-cb894aa8b794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:56.598417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2818d5-2a22-450b-ab8d-cb894aa8b794 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:56.616858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-2e2818d5-2a22-450b-ab8d-cb894aa8b794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:56.616858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:56.616858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:56.616858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:56.616858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:56.616858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:56.616858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:56.616858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:56.616858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:56.616858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:56.616858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:56.616858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:56.616858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:56.616858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:56.616858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:56.617933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:56.617933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:56.617933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:56.617933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:56.617933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:56.617933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:56.617933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:56.617933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:56.617933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:56.617933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:56.617933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:56.617933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:56.617933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:56.617933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:56.617933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:56.617933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:56.617933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:56.623895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:56.623895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:56.623895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:56.623895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:56.623895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 324/653] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:56.624241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92dcf31-0bd7-4e00-aa45-b66d293a22eb None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:56.624338 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2818d5-2a22-450b-ab8d-cb894aa8b794 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:56.661248 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc44dcd-3219-466b-8eda-239168af6597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:56.661873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc44dcd-3219-466b-8eda-239168af6597 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:56.662150 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc44dcd-3219-466b-8eda-239168af6597 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:56.662420 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc44dcd-3219-466b-8eda-239168af6597 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:56.663116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc44dcd-3219-466b-8eda-239168af6597 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:56.663415 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc44dcd-3219-466b-8eda-239168af6597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:56.664847 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc44dcd-3219-466b-8eda-239168af6597 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:56.681051 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 330/654] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:56.681051 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc44dcd-3219-466b-8eda-239168af6597 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:56.706532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:56.745395 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2818d5-2a22-450b-ab8d-cb894aa8b794 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:56.761565 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:56.778933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc44dcd-3219-466b-8eda-239168af6597 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:56.784226 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3ff3a-8cb9-409a-819f-2f23e4b268d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:56.784891 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3ff3a-8cb9-409a-819f-2f23e4b268d2 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:56.785170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3ff3a-8cb9-409a-819f-2f23e4b268d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:56.785439 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3ff3a-8cb9-409a-819f-2f23e4b268d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:56.786330 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3ff3a-8cb9-409a-819f-2f23e4b268d2 None admin] RBAC: Adding query filter params (role.id=977824a8c6a24c00af2f6dabd0e2a6f0, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=c571406d2a3f40dab035a918399d6f6a) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:56.786640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3ff3a-8cb9-409a-819f-2f23e4b268d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:56.788072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3ff3a-8cb9-409a-819f-2f23e4b268d2 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:56.804807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 325/655] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:56 2022] GET /identity/v3/role_assignments?user.id=c571406d2a3f40dab035a918399d6f6a&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=977824a8c6a24c00af2f6dabd0e2a6f0 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:56.804807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3ff3a-8cb9-409a-819f-2f23e4b268d2 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:56.832770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557c09f-42fa-4612-b704-ec09331f1c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:56.833324 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557c09f-42fa-4612-b704-ec09331f1c3b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:56.833604 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557c09f-42fa-4612-b704-ec09331f1c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:56.833873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557c09f-42fa-4612-b704-ec09331f1c3b None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:56.834575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557c09f-42fa-4612-b704-ec09331f1c3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:56.834875 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557c09f-42fa-4612-b704-ec09331f1c3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:56.836270 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f557c09f-42fa-4612-b704-ec09331f1c3b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:56.849949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 331/656] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:56.850823 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557c09f-42fa-4612-b704-ec09331f1c3b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:56.941072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3ff3a-8cb9-409a-819f-2f23e4b268d2 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:56.974903 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:56.980139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528916c6-ac00-41cc-97be-dc5701b6943e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:56.980812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528916c6-ac00-41cc-97be-dc5701b6943e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:56.981094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528916c6-ac00-41cc-97be-dc5701b6943e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:56.981366 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528916c6-ac00-41cc-97be-dc5701b6943e None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:56.982054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528916c6-ac00-41cc-97be-dc5701b6943e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:56.982355 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528916c6-ac00-41cc-97be-dc5701b6943e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:56.983742 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528916c6-ac00-41cc-97be-dc5701b6943e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:56.990533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557c09f-42fa-4612-b704-ec09331f1c3b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:56.997441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 326/657] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:56.998095 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528916c6-ac00-41cc-97be-dc5701b6943e None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:57.035838 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7814cc-870c-4152-8923-420232b73794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:57.036470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7814cc-870c-4152-8923-420232b73794 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:57.036774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7814cc-870c-4152-8923-420232b73794 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:57.037045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7814cc-870c-4152-8923-420232b73794 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:57.046564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:57.046877 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7814cc-870c-4152-8923-420232b73794 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:57.048962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7814cc-870c-4152-8923-420232b73794 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:57.064820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-cf7814cc-870c-4152-8923-420232b73794 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 02:04:57.064820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:57.064820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:57.064820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:57.064820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:57.064820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:57.064820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:57.064820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:57.064820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:57.064820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:57.064820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:57.064820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:57.064820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:04:57.064820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:04:57.064820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:04:57.064820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:57.065803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:57.067075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:57.067075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:57.067075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:04:57.067075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:04:57.067075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:04:57.067075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:04:57.067075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:04:57.067075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:04:57.067075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 04 02:04:57.067075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:57.067075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 332/658] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:57.067803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f3b74-dc07-47ef-a6d3-9f623c2764e3 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:57.067803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f3b74-dc07-47ef-a6d3-9f623c2764e3 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:57.067803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f3b74-dc07-47ef-a6d3-9f623c2764e3 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:57.068942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 333/659] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:57.069638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f3b74-dc07-47ef-a6d3-9f623c2764e3 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:57.119114 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528916c6-ac00-41cc-97be-dc5701b6943e None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:57.157824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e778ab9b-27d3-4a21-a483-e7feb52fb502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:57.158335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778ab9b-27d3-4a21-a483-e7feb52fb502 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:57.158613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778ab9b-27d3-4a21-a483-e7feb52fb502 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:57.158881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e778ab9b-27d3-4a21-a483-e7feb52fb502 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:57.169835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e778ab9b-27d3-4a21-a483-e7feb52fb502 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:57.171485 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e778ab9b-27d3-4a21-a483-e7feb52fb502 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:57.179838 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f3b74-dc07-47ef-a6d3-9f623c2764e3 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:57.185615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-e778ab9b-27d3-4a21-a483-e7feb52fb502 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 02:04:57.185615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:57.185615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:57.185615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:57.185615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:57.185615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:57.185615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:57.185615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:57.185615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:57.185615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:57.185615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:57.185615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:57.185615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:04:57.185615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:04:57.185615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:04:57.185615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:57.186552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:57.187792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:57.187792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:57.187792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:04:57.187792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:04:57.187792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:04:57.187792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:04:57.187792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:04:57.187792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:04:57.187792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 02:04:57.187792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:57.191836 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 327/660] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:57.193130 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ce713-33f9-4d28-8430-7b00bde736d4 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:57.193419 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ce713-33f9-4d28-8430-7b00bde736d4 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:57.193697 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ce713-33f9-4d28-8430-7b00bde736d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:57.212469 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-ea3ce713-33f9-4d28-8430-7b00bde736d4 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:57.213755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ea3ce713-33f9-4d28-8430-7b00bde736d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:57.218530 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4eb64-9e75-41ef-8b1d-b0948967483d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:57.219199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4eb64-9e75-41ef-8b1d-b0948967483d None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:57.219446 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4eb64-9e75-41ef-8b1d-b0948967483d None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:57.219709 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4eb64-9e75-41ef-8b1d-b0948967483d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:57.227796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4eb64-9e75-41ef-8b1d-b0948967483d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:57.229260 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4eb64-9e75-41ef-8b1d-b0948967483d None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:57.241222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-67b4eb64-9e75-41ef-8b1d-b0948967483d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:57.241222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:57.241222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:57.241222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:57.241222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:57.241222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:57.241222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:57.241222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:57.241222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:57.241222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:57.241222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:57.241222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:57.241222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:04:57.241222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:04:57.241222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:04:57.242378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:04:57.242378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:57.242378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:57.242378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:57.242378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:57.242378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:57.242378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:57.242378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:57.242378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:57.242378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:57.242378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:57.242378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:57.242378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:57.242378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:57.242378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:57.242378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:57.242378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:57.243474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:04:57.243474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:04:57.243474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:04:57.243474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:04:57.243474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:04:57.243474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:04:57.243474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:57.243474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:57.245269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 334/661] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:57.246254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4eb64-9e75-41ef-8b1d-b0948967483d None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:57.258390 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:57.293401 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 328/662] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:57.294071 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ce713-33f9-4d28-8430-7b00bde736d4 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:57.322807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:57.366243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4eb64-9e75-41ef-8b1d-b0948967483d None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:57.399334 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3ce713-33f9-4d28-8430-7b00bde736d4 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:57.403853 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22027e1b-3b01-4624-b2d3-a8e0afc30ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:57.404476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22027e1b-3b01-4624-b2d3-a8e0afc30ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:57.404796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22027e1b-3b01-4624-b2d3-a8e0afc30ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:57.405073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22027e1b-3b01-4624-b2d3-a8e0afc30ed1 None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:57.405870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22027e1b-3b01-4624-b2d3-a8e0afc30ed1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:57.406176 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22027e1b-3b01-4624-b2d3-a8e0afc30ed1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:57.407607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22027e1b-3b01-4624-b2d3-a8e0afc30ed1 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:57.430587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 335/663] 10.222.0.45 () {56 vars in 1139 bytes} [Thu Aug 4 02:04:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:57.430587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22027e1b-3b01-4624-b2d3-a8e0afc30ed1 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:57.440171 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5ce60-fe30-4d2c-aba5-139a82f3f249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:57.440820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5ce60-fe30-4d2c-aba5-139a82f3f249 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:57.441097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5ce60-fe30-4d2c-aba5-139a82f3f249 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:57.441375 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5ce60-fe30-4d2c-aba5-139a82f3f249 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:57.442171 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5ce60-fe30-4d2c-aba5-139a82f3f249 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:57.442485 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5ce60-fe30-4d2c-aba5-139a82f3f249 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:57.443899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5ce60-fe30-4d2c-aba5-139a82f3f249 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:57.467751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 329/664] 10.222.0.45 () {56 vars in 1137 bytes} [Thu Aug 4 02:04:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:57.467751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5ce60-fe30-4d2c-aba5-139a82f3f249 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:57.504408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:57.546456 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22027e1b-3b01-4624-b2d3-a8e0afc30ed1 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:57.568955 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:57.585558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5ce60-fe30-4d2c-aba5-139a82f3f249 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:57.585739 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02fdb3c-df4e-40e4-96b9-88d5bc2f80f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:57.586360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fdb3c-df4e-40e4-96b9-88d5bc2f80f3 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:57.586638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fdb3c-df4e-40e4-96b9-88d5bc2f80f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:57.586914 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02fdb3c-df4e-40e4-96b9-88d5bc2f80f3 None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:57.587661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02fdb3c-df4e-40e4-96b9-88d5bc2f80f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:57.588608 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02fdb3c-df4e-40e4-96b9-88d5bc2f80f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:57.589410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02fdb3c-df4e-40e4-96b9-88d5bc2f80f3 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:57.606838 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 336/665] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:04:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:57.606838 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02fdb3c-df4e-40e4-96b9-88d5bc2f80f3 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:57.625978 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5182e05-b6bb-4997-8057-2d8d2fe97453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:57.626618 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5182e05-b6bb-4997-8057-2d8d2fe97453 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:57.626907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5182e05-b6bb-4997-8057-2d8d2fe97453 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:57.627176 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5182e05-b6bb-4997-8057-2d8d2fe97453 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:57.638178 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5182e05-b6bb-4997-8057-2d8d2fe97453 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:57.642976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-b5182e05-b6bb-4997-8057-2d8d2fe97453 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:57.642976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:57.642976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:57.642976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:57.642976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:57.642976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:57.642976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:57.642976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:57.642976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:57.642976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:57.642976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:57.642976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:57.642976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:57.642976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:57.642976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:57.642976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:57.644509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:57.644509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:57.644509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:57.644509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:57.644509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:57.644509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:57.644509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:57.644509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:57.644509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:57.644509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:57.644509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:57.644509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:57.644509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:57.644509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:57.644509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:57.644509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:57.645831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:57.645831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:57.645831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:57.645831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:57.645831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:57.645831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:57.645831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:57.645831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:57.645831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:57.645831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:57.645831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:04:57.645831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:57.648235 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 330/666] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:04:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:57.649192 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5182e05-b6bb-4997-8057-2d8d2fe97453 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:57.724876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02fdb3c-df4e-40e4-96b9-88d5bc2f80f3 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:57.752483 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:57.758775 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5182e05-b6bb-4997-8057-2d8d2fe97453 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:57.763231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d64304-3e65-43c4-85f1-74fafff7a31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:57.763859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d64304-3e65-43c4-85f1-74fafff7a31c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:57.764142 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d64304-3e65-43c4-85f1-74fafff7a31c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:57.764412 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d64304-3e65-43c4-85f1-74fafff7a31c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:57.794919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dcc31d-5bd8-420c-9cac-4f954f56a2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:57.795783 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcc31d-5bd8-420c-9cac-4f954f56a2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:57.796053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcc31d-5bd8-420c-9cac-4f954f56a2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:57.796323 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dcc31d-5bd8-420c-9cac-4f954f56a2c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:57.811321 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:57.839122 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d64304-3e65-43c4-85f1-74fafff7a31c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:57.840564 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d64304-3e65-43c4-85f1-74fafff7a31c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:57.866022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dcc31d-5bd8-420c-9cac-4f954f56a2c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:57.867466 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dcc31d-5bd8-420c-9cac-4f954f56a2c7 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:57.946972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 337/667] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:57.948021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d64304-3e65-43c4-85f1-74fafff7a31c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:57.983174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:57.992043 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 331/668] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:04:57.992911 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dcc31d-5bd8-420c-9cac-4f954f56a2c7 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:58.045057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:58.080534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d64304-3e65-43c4-85f1-74fafff7a31c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:58.104276 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dcc31d-5bd8-420c-9cac-4f954f56a2c7 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:58.118227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d06a1e-afb3-4670-b790-5e21b1e6ac27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:58.119204 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d06a1e-afb3-4670-b790-5e21b1e6ac27 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:58.119484 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d06a1e-afb3-4670-b790-5e21b1e6ac27 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:58.119899 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d06a1e-afb3-4670-b790-5e21b1e6ac27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:58.120710 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d06a1e-afb3-4670-b790-5e21b1e6ac27 None admin] RBAC: Adding query filter params (role.id=977824a8c6a24c00af2f6dabd0e2a6f0, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=14ff31d52ecf4c7d93a65bbfca7285c0) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:58.121020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d06a1e-afb3-4670-b790-5e21b1e6ac27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:58.122429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d06a1e-afb3-4670-b790-5e21b1e6ac27 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:58.136664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 338/669] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:57 2022] GET /identity/v3/role_assignments?user.id=14ff31d52ecf4c7d93a65bbfca7285c0&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=977824a8c6a24c00af2f6dabd0e2a6f0 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:58.136664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d06a1e-afb3-4670-b790-5e21b1e6ac27 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:58.157408 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9317b-bcbd-4e10-a6f1-f2f011f7ec50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:58.158075 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9317b-bcbd-4e10-a6f1-f2f011f7ec50 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:58.158422 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9317b-bcbd-4e10-a6f1-f2f011f7ec50 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:58.158696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9317b-bcbd-4e10-a6f1-f2f011f7ec50 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:58.160000 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9317b-bcbd-4e10-a6f1-f2f011f7ec50 None admin] RBAC: Adding query filter params (name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:58.160300 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9317b-bcbd-4e10-a6f1-f2f011f7ec50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:58.161873 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9317b-bcbd-4e10-a6f1-f2f011f7ec50 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:58.180420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 332/670] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:04:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:58.180420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9317b-bcbd-4e10-a6f1-f2f011f7ec50 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:58.270478 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:58.282520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d06a1e-afb3-4670-b790-5e21b1e6ac27 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:58.296931 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9317b-bcbd-4e10-a6f1-f2f011f7ec50 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:58.306508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:58.329981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a44bbc-6194-4142-8137-a9acb87cfd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:58.330713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a44bbc-6194-4142-8137-a9acb87cfd7f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:58.331026 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a44bbc-6194-4142-8137-a9acb87cfd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:58.331297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a44bbc-6194-4142-8137-a9acb87cfd7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:58.337317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d2acd-61a3-44c7-890e-dd5a65e5d365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:58.338206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d2acd-61a3-44c7-890e-dd5a65e5d365 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:58.338660 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d2acd-61a3-44c7-890e-dd5a65e5d365 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:58.339126 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549d2acd-61a3-44c7-890e-dd5a65e5d365 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:58.344202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a44bbc-6194-4142-8137-a9acb87cfd7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:58.345910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a44bbc-6194-4142-8137-a9acb87cfd7f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:58.353288 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d2acd-61a3-44c7-890e-dd5a65e5d365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:58.354734 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549d2acd-61a3-44c7-890e-dd5a65e5d365 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:58.367165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-e2a44bbc-6194-4142-8137-a9acb87cfd7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:58.367165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:58.367165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:58.367165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:58.367165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:58.367165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:58.367165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:58.367165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:58.367165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:58.367165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:58.367165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:58.367165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:58.367165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:58.367165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:58.367165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:58.368169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:58.368169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:58.368169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:58.368169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:58.368169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:58.368169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:58.368169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:58.368169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:58.368169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:58.368169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:58.368169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:58.368169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:58.368169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:58.368169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:58.368169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:58.368169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:58.368169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:58.369216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:58.369216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:58.369216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:58.369216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:58.369216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 339/671] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:58.369702 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a44bbc-6194-4142-8137-a9acb87cfd7f None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:58.375034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-549d2acd-61a3-44c7-890e-dd5a65e5d365 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:58.375034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:58.375034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:58.375034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:58.375034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:58.375034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:58.375034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:58.375034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:58.375034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:58.375034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:58.375034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:58.375034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:58.375034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:58.375034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:58.375034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:58.375958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:58.375958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:58.375958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:58.375958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:58.375958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:58.375958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:58.375958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:58.375958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:58.375958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:58.375958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:58.375958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:58.375958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:58.375958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:58.375958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:58.375958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:58.375958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:58.375958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:58.376950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:58.376950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:58.376950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:58.376950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:58.380871 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 333/672] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:58.381773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d2acd-61a3-44c7-890e-dd5a65e5d365 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:58.496873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a44bbc-6194-4142-8137-a9acb87cfd7f None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:58.502858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549d2acd-61a3-44c7-890e-dd5a65e5d365 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:58.532527 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab3921-776d-4790-aadd-faa424bdc593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:58.533115 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab3921-776d-4790-aadd-faa424bdc593 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:58.533650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab3921-776d-4790-aadd-faa424bdc593 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:58.533650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab3921-776d-4790-aadd-faa424bdc593 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:58.542271 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ded23-7228-4561-b4d8-7daa7b7ac78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:58.543191 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ded23-7228-4561-b4d8-7daa7b7ac78e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:58.543700 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ded23-7228-4561-b4d8-7daa7b7ac78e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:58.544178 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ded23-7228-4561-b4d8-7daa7b7ac78e None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:58.545106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ded23-7228-4561-b4d8-7daa7b7ac78e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:58.545641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ded23-7228-4561-b4d8-7daa7b7ac78e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:58.546216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab3921-776d-4790-aadd-faa424bdc593 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:58.547471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ded23-7228-4561-b4d8-7daa7b7ac78e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:58.547664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab3921-776d-4790-aadd-faa424bdc593 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:58.549656 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:58.563032 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 334/673] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:58.564114 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ded23-7228-4561-b4d8-7daa7b7ac78e None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:58.565243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-08ab3921-776d-4790-aadd-faa424bdc593 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:58.565243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:58.565243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:58.565243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:58.565243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:58.565243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:58.565243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:58.565243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:58.565243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:58.565243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:58.565243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:58.565243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:58.565243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:04:58.565243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:04:58.565243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:04:58.566199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:04:58.566199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:58.566199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:58.566199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:58.566199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:58.566199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:58.566199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:58.566199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:58.566199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:58.566199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:58.566199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:58.566199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:58.566199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:58.566199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:58.566199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:58.566199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:58.566199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:58.567080 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:04:58.567080 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:04:58.567080 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:04:58.567080 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:58.569222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 340/674] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:58.574661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab3921-776d-4790-aadd-faa424bdc593 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:58.588927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:58.682195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ded23-7228-4561-b4d8-7daa7b7ac78e None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:58.688435 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab3921-776d-4790-aadd-faa424bdc593 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:58.719096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900ca26-eac7-4c1d-8bbc-b2c74085f2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:58.720105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900ca26-eac7-4c1d-8bbc-b2c74085f2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:58.720749 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900ca26-eac7-4c1d-8bbc-b2c74085f2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:58.721275 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900ca26-eac7-4c1d-8bbc-b2c74085f2d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:58.722849 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900ca26-eac7-4c1d-8bbc-b2c74085f2d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:58.723503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900ca26-eac7-4c1d-8bbc-b2c74085f2d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:58.724601 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12338d8e-a6f3-4a0c-aae9-67d463a81bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:58.725232 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900ca26-eac7-4c1d-8bbc-b2c74085f2d4 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:58.725347 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12338d8e-a6f3-4a0c-aae9-67d463a81bab None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:58.725619 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12338d8e-a6f3-4a0c-aae9-67d463a81bab None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:58.725886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12338d8e-a6f3-4a0c-aae9-67d463a81bab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:58.733180 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12338d8e-a6f3-4a0c-aae9-67d463a81bab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:58.734614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12338d8e-a6f3-4a0c-aae9-67d463a81bab None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:58.738168 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 335/675] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:58.739110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900ca26-eac7-4c1d-8bbc-b2c74085f2d4 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:58.746452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-12338d8e-a6f3-4a0c-aae9-67d463a81bab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:58.746452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:58.746452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:58.746452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:58.746452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:58.746452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:58.746452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:58.746452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:58.746452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:58.746452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:58.746452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:58.746452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:58.746452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:04:58.746452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:04:58.746452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:04:58.747387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:04:58.747387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:58.747387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:58.747387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:58.747387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:58.747387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:58.747387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:58.747387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:58.747387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:58.747387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:58.747387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:58.747387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:58.747387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:58.747387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:58.747387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:58.747387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:58.747387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:58.748269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:04:58.748269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:04:58.748269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:04:58.748269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:04:58.748269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:04:58.748269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:04:58.748269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:58.748269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:58.748269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 341/676] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:58.749339 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12338d8e-a6f3-4a0c-aae9-67d463a81bab None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:58.806309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:58.826736 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:58.848924 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900ca26-eac7-4c1d-8bbc-b2c74085f2d4 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:58.853444 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12338d8e-a6f3-4a0c-aae9-67d463a81bab None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:58.888072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf77ef-8bf0-43df-add3-09b019d85a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:58.888311 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba92f4b-83f8-4e38-934b-fa7d5d85a551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:58.888995 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba92f4b-83f8-4e38-934b-fa7d5d85a551 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:58.889277 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba92f4b-83f8-4e38-934b-fa7d5d85a551 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:58.889449 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf77ef-8bf0-43df-add3-09b019d85a44 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:58.889561 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba92f4b-83f8-4e38-934b-fa7d5d85a551 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:58.889957 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf77ef-8bf0-43df-add3-09b019d85a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:58.890235 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba92f4b-83f8-4e38-934b-fa7d5d85a551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:58.890540 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba92f4b-83f8-4e38-934b-fa7d5d85a551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:58.890653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf77ef-8bf0-43df-add3-09b019d85a44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:58.892132 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba92f4b-83f8-4e38-934b-fa7d5d85a551 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:58.898506 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf77ef-8bf0-43df-add3-09b019d85a44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:58.900214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaf77ef-8bf0-43df-add3-09b019d85a44 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:58.910234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 336/677] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:58.910234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba92f4b-83f8-4e38-934b-fa7d5d85a551 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:58.919513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-dbaf77ef-8bf0-43df-add3-09b019d85a44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:58.919513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:58.919513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:58.919513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:58.919513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:58.919513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:58.919513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:58.919513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:58.919513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:58.919513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:58.919513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:58.919513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:58.919513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:04:58.919513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:04:58.919513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:04:58.922629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:04:58.922629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:58.922629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:58.922629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:58.922629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:58.922629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:58.922629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:58.922629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:58.922629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:58.922629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:58.922629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:58.922629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:58.922629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:58.922629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:58.922629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:58.922629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:58.922629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:58.923511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:04:58.923511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:04:58.923511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:04:58.923511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:04:58.923511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:04:58.923511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:04:58.923511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:04:58.923511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:58.928795 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 342/678] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:04:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:58.929941 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf77ef-8bf0-43df-add3-09b019d85a44 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:59.016965 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba92f4b-83f8-4e38-934b-fa7d5d85a551 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:59.050552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf77ef-8bf0-43df-add3-09b019d85a44 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:59.052267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959c8af-2781-4d8f-84ed-be04cc19b32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:59.052914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959c8af-2781-4d8f-84ed-be04cc19b32e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:59.053194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959c8af-2781-4d8f-84ed-be04cc19b32e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:59.053462 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959c8af-2781-4d8f-84ed-be04cc19b32e None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:59.054202 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959c8af-2781-4d8f-84ed-be04cc19b32e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:59.054503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959c8af-2781-4d8f-84ed-be04cc19b32e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:59.055919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959c8af-2781-4d8f-84ed-be04cc19b32e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:59.065093 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:59.068878 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:59.069776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 337/679] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:04:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:59.070584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959c8af-2781-4d8f-84ed-be04cc19b32e None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:59.097085 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b386a6e-0be2-4c2f-b21c-f593678f05de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:59.097725 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b386a6e-0be2-4c2f-b21c-f593678f05de None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:59.098007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b386a6e-0be2-4c2f-b21c-f593678f05de None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:59.098287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b386a6e-0be2-4c2f-b21c-f593678f05de None admin] PATH_INFO: `/v3/users/swift` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:59.106379 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b386a6e-0be2-4c2f-b21c-f593678f05de None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:59.107825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b386a6e-0be2-4c2f-b21c-f593678f05de None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:59.120351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-0b386a6e-0be2-4c2f-b21c-f593678f05de None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 02:04:59.120351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:59.120351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:59.120351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:59.120351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:59.120351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:59.120351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:59.120351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:59.120351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:59.120351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:59.120351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:59.120351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:59.120351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:04:59.120351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:04:59.120351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:04:59.120351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:59.121348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:59.122312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:59.122312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:59.122312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:04:59.122312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:04:59.122312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:04:59.122312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:04:59.122312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:04:59.122312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:04:59.122312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 02:04:59.122312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:04:59.122312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 343/680] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:04:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:59.122940 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9a8dd-8bd7-4c7d-a072-07f4ddcc351d None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:59.123094 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9a8dd-8bd7-4c7d-a072-07f4ddcc351d None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:59.123503 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9a8dd-8bd7-4c7d-a072-07f4ddcc351d None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:59.124603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 344/681] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:04:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:04:59.129350 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9a8dd-8bd7-4c7d-a072-07f4ddcc351d None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:59.184316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959c8af-2781-4d8f-84ed-be04cc19b32e None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:59.220173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79405c7-f4ac-4ea3-923a-2e85c32955b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:59.220818 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79405c7-f4ac-4ea3-923a-2e85c32955b9 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:59.221099 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79405c7-f4ac-4ea3-923a-2e85c32955b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:59.221372 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79405c7-f4ac-4ea3-923a-2e85c32955b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:59.222649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79405c7-f4ac-4ea3-923a-2e85c32955b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:59.222956 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79405c7-f4ac-4ea3-923a-2e85c32955b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:59.224381 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79405c7-f4ac-4ea3-923a-2e85c32955b9 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:59.236244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 338/682] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:04:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:59.237126 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79405c7-f4ac-4ea3-923a-2e85c32955b9 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:59.243629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9a8dd-8bd7-4c7d-a072-07f4ddcc351d None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:59.281503 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4f1c0b-b606-4975-98b9-7584809fc547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:59.282116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f1c0b-b606-4975-98b9-7584809fc547 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:59.282396 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f1c0b-b606-4975-98b9-7584809fc547 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:59.282668 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4f1c0b-b606-4975-98b9-7584809fc547 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:59.283462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f1c0b-b606-4975-98b9-7584809fc547 None admin] RBAC: Adding query filter params (role.id=977824a8c6a24c00af2f6dabd0e2a6f0, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=6c2da427acb24c8cb6d667c6347858c7) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:59.283770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f1c0b-b606-4975-98b9-7584809fc547 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:59.285757 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4f1c0b-b606-4975-98b9-7584809fc547 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:59.299377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 345/683] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:59 2022] GET /identity/v3/role_assignments?user.id=6c2da427acb24c8cb6d667c6347858c7&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=977824a8c6a24c00af2f6dabd0e2a6f0 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:59.299377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70972aa2-17ec-439e-b958-c4e6e40e36ef None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:59.299377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70972aa2-17ec-439e-b958-c4e6e40e36ef None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:59.299377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70972aa2-17ec-439e-b958-c4e6e40e36ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:59.314102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-70972aa2-17ec-439e-b958-c4e6e40e36ef None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:04:59.315522 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-70972aa2-17ec-439e-b958-c4e6e40e36ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:04:59.323886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:59.358188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:59.376878 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79405c7-f4ac-4ea3-923a-2e85c32955b9 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:59.416586 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18851b-eef7-4e0b-b81a-89f5e2ceb6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:59.417249 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18851b-eef7-4e0b-b81a-89f5e2ceb6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:59.417523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18851b-eef7-4e0b-b81a-89f5e2ceb6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:59.417791 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18851b-eef7-4e0b-b81a-89f5e2ceb6d6 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:59.418593 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18851b-eef7-4e0b-b81a-89f5e2ceb6d6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:59.418893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18851b-eef7-4e0b-b81a-89f5e2ceb6d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:59.420335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18851b-eef7-4e0b-b81a-89f5e2ceb6d6 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:59.421047 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 346/684] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:04:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:04:59.422022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70972aa2-17ec-439e-b958-c4e6e40e36ef None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:59.455618 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 339/685] 10.222.0.45 () {56 vars in 1135 bytes} [Thu Aug 4 02:04:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:59.457257 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18851b-eef7-4e0b-b81a-89f5e2ceb6d6 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:59.567903 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70972aa2-17ec-439e-b958-c4e6e40e36ef None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:59.591501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18851b-eef7-4e0b-b81a-89f5e2ceb6d6 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:59.604539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86b98d-bc43-45fc-85a8-e5dae6a6a565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:59.605283 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86b98d-bc43-45fc-85a8-e5dae6a6a565 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:59.605560 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86b98d-bc43-45fc-85a8-e5dae6a6a565 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:59.605834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86b98d-bc43-45fc-85a8-e5dae6a6a565 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:59.606652 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa86b98d-bc43-45fc-85a8-e5dae6a6a565 None admin] RBAC: Adding query filter params (role.id=977824a8c6a24c00af2f6dabd0e2a6f0, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=f4b8838129174fbb8cdd32bd5d4d6101) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:59.606958 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa86b98d-bc43-45fc-85a8-e5dae6a6a565 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:59.608373 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa86b98d-bc43-45fc-85a8-e5dae6a6a565 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:59.613866 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:59.614736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 347/686] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:04:59 2022] GET /identity/v3/role_assignments?user.id=f4b8838129174fbb8cdd32bd5d4d6101&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=977824a8c6a24c00af2f6dabd0e2a6f0 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:59.615758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86b98d-bc43-45fc-85a8-e5dae6a6a565 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:59.636717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777979f-6f37-4319-80ff-dfc7d47a5933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:59.637784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777979f-6f37-4319-80ff-dfc7d47a5933 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:59.638074 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777979f-6f37-4319-80ff-dfc7d47a5933 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:59.638357 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777979f-6f37-4319-80ff-dfc7d47a5933 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:59.664525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777979f-6f37-4319-80ff-dfc7d47a5933 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:04:59.669429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-2777979f-6f37-4319-80ff-dfc7d47a5933 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:04:59.669429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:04:59.669429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:04:59.669429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:04:59.669429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:04:59.669429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:04:59.669429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:04:59.669429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:04:59.669429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:04:59.669429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:04:59.669429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:04:59.669429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:04:59.669429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:04:59.669429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:04:59.669429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:04:59.669429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:04:59.670360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:04:59.670360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:04:59.670360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:04:59.670360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:04:59.670360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:04:59.670360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:04:59.670360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:04:59.670360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:04:59.670360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:04:59.670360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:04:59.670360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:04:59.670360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:04:59.670360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:04:59.670360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:04:59.670360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:04:59.670360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:04:59.671601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:04:59.671601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:04:59.671601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:04:59.671601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:04:59.671601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:04:59.671601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:04:59.671601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:04:59.671601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:04:59.671601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:04:59.671601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:04:59.671601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:04:59.671601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:04:59.671601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 340/687] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:04:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:04:59.675975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:04:59.677681 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777979f-6f37-4319-80ff-dfc7d47a5933 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:59.764695 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86b98d-bc43-45fc-85a8-e5dae6a6a565 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:59.810774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb075f-c86c-4367-928b-0df71c451762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:59.811800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb075f-c86c-4367-928b-0df71c451762 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:59.812536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb075f-c86c-4367-928b-0df71c451762 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:59.813035 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cb075f-c86c-4367-928b-0df71c451762 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:59.869451 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777979f-6f37-4319-80ff-dfc7d47a5933 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:04:59.922919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af844782-1780-4805-9baf-873f0cd837ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:04:59.923818 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af844782-1780-4805-9baf-873f0cd837ee None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:04:59.924144 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb075f-c86c-4367-928b-0df71c451762 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:59.924427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af844782-1780-4805-9baf-873f0cd837ee None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:04:59.924910 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af844782-1780-4805-9baf-873f0cd837ee None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:04:59.925802 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af844782-1780-4805-9baf-873f0cd837ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:04:59.926010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cb075f-c86c-4367-928b-0df71c451762 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:59.926405 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af844782-1780-4805-9baf-873f0cd837ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:04:59.927925 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af844782-1780-4805-9baf-873f0cd837ee None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:04:59.938558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 341/688] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:04:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:04:59.946784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af844782-1780-4805-9baf-873f0cd837ee None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:04:59.962556 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:00.009210 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:00.055215 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 348/689] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:04:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:05:00.061096 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb075f-c86c-4367-928b-0df71c451762 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:00.087604 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af844782-1780-4805-9baf-873f0cd837ee None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:00.132693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577271d0-0ab5-42f5-abaf-29067283c396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:00.133284 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577271d0-0ab5-42f5-abaf-29067283c396 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:00.133703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577271d0-0ab5-42f5-abaf-29067283c396 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:00.133996 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577271d0-0ab5-42f5-abaf-29067283c396 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:00.157182 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577271d0-0ab5-42f5-abaf-29067283c396 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:00.158981 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577271d0-0ab5-42f5-abaf-29067283c396 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:00.187735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-577271d0-0ab5-42f5-abaf-29067283c396 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:00.187735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:00.187735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:00.187735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:00.187735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:00.187735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:00.187735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:00.187735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:00.187735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:00.187735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:00.187735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:00.187735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:00.187735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:00.187735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:00.187735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:00.188708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:00.188708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:00.188708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:00.188708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:00.188708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:00.188708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:00.188708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:00.188708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:00.188708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:00.188708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:00.188708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:00.188708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:00.188708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:00.188708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:00.188708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:00.188708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:00.188708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:00.189605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:00.189605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:00.189605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:00.189605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:00.192408 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 342/690] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:04:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:00.201571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577271d0-0ab5-42f5-abaf-29067283c396 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:00.221360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cb075f-c86c-4367-928b-0df71c451762 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:00.253751 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:00.258736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5e027-98fb-4772-b6db-357bd963a8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:00.259391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5e027-98fb-4772-b6db-357bd963a8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:00.259679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5e027-98fb-4772-b6db-357bd963a8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:00.260020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5e027-98fb-4772-b6db-357bd963a8e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:00.271429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5e027-98fb-4772-b6db-357bd963a8e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:00.272934 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5e027-98fb-4772-b6db-357bd963a8e5 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:00.292143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-62b5e027-98fb-4772-b6db-357bd963a8e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:00.292143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:00.292143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:00.292143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:00.292143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:00.292143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:00.292143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:00.292143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:00.292143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:00.292143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:00.292143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:00.292143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:00.292143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:00.292143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:00.292143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:00.293174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:00.293174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:00.293174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:00.293174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:00.293174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:00.293174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:00.293174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:00.293174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:00.293174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:00.293174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:00.293174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:00.293174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:00.293174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:00.293174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:00.293174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:00.293174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:00.293174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:00.294075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:00.294075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:00.294075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:00.294075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:00.294075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 349/691] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:00.299167 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5e027-98fb-4772-b6db-357bd963a8e5 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:00.403986 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5e027-98fb-4772-b6db-357bd963a8e5 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:00.439882 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fcc253-521f-416b-b0fc-32e08aba1e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:00.441193 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcc253-521f-416b-b0fc-32e08aba1e3f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:00.441486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcc253-521f-416b-b0fc-32e08aba1e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:00.441762 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fcc253-521f-416b-b0fc-32e08aba1e3f None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:00.442678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fcc253-521f-416b-b0fc-32e08aba1e3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:00.442786 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fcc253-521f-416b-b0fc-32e08aba1e3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:00.444110 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fcc253-521f-416b-b0fc-32e08aba1e3f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:00.457425 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 350/692] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:00.457755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:00.462286 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f6cb3-c007-4f20-9117-3f9d63c537a5 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:00.462656 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f6cb3-c007-4f20-9117-3f9d63c537a5 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:00.462847 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f6cb3-c007-4f20-9117-3f9d63c537a5 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:00.464276 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 351/693] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:00.466377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f6cb3-c007-4f20-9117-3f9d63c537a5 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:00.475212 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577271d0-0ab5-42f5-abaf-29067283c396 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:00.514891 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4c184-dfc1-4b0b-b4c4-1c57ce19ccd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:00.516153 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4c184-dfc1-4b0b-b4c4-1c57ce19ccd4 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:00.516637 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4c184-dfc1-4b0b-b4c4-1c57ce19ccd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:00.517149 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:00.517398 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4c184-dfc1-4b0b-b4c4-1c57ce19ccd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:00.518665 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4c184-dfc1-4b0b-b4c4-1c57ce19ccd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:00.519421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4c184-dfc1-4b0b-b4c4-1c57ce19ccd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:00.521458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c4c184-dfc1-4b0b-b4c4-1c57ce19ccd4 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:00.537078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 343/694] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:00.538273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cc487-ff12-4690-8efc-7c07ab645fb1 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:00.538561 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cc487-ff12-4690-8efc-7c07ab645fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:00.538840 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cc487-ff12-4690-8efc-7c07ab645fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:00.562993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-642cc487-ff12-4690-8efc-7c07ab645fb1 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:00.564198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-642cc487-ff12-4690-8efc-7c07ab645fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:00.583050 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f6cb3-c007-4f20-9117-3f9d63c537a5 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:00.626216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5177b6-293a-4df0-b0fc-110850ddc30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:00.627274 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5177b6-293a-4df0-b0fc-110850ddc30b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:00.627859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5177b6-293a-4df0-b0fc-110850ddc30b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:00.628316 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5177b6-293a-4df0-b0fc-110850ddc30b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:00.639374 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5177b6-293a-4df0-b0fc-110850ddc30b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:00.642299 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5177b6-293a-4df0-b0fc-110850ddc30b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:00.678608 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 344/695] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:05:00.679486 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cc487-ff12-4690-8efc-7c07ab645fb1 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:00.685202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-1d5177b6-293a-4df0-b0fc-110850ddc30b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:00.685202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:00.685202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:00.685202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:00.685202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:00.685202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:00.685202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:00.685202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:00.685202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:00.685202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:00.685202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:00.685202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:00.685202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:05:00.685202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:05:00.685202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:05:00.686132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:05:00.686132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:00.686132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:00.686132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:00.686132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:00.686132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:00.686132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:00.686132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:00.686132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:00.686132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:00.686132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:00.686132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:00.686132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:00.686132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:00.686132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:00.686132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:00.686132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:00.687808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:05:00.687808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:05:00.687808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:05:00.687808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:05:00.687808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:05:00.687808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:05:00.687808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:00.687808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:00.696309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 352/696] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:00.697760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5177b6-293a-4df0-b0fc-110850ddc30b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:00.809569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:00.828134 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cc487-ff12-4690-8efc-7c07ab645fb1 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:00.830354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5177b6-293a-4df0-b0fc-110850ddc30b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:00.836335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:00.864415 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41419d2-0edf-4f0a-93ef-5a817a60a12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:00.865108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41419d2-0edf-4f0a-93ef-5a817a60a12e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:00.865387 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41419d2-0edf-4f0a-93ef-5a817a60a12e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:00.865659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41419d2-0edf-4f0a-93ef-5a817a60a12e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:00.874859 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41419d2-0edf-4f0a-93ef-5a817a60a12e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:00.874859 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41419d2-0edf-4f0a-93ef-5a817a60a12e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:00.883750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589ddd2-e13e-4e12-8e3f-387c4b18e525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:00.884605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589ddd2-e13e-4e12-8e3f-387c4b18e525 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:00.885099 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589ddd2-e13e-4e12-8e3f-387c4b18e525 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:00.885553 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589ddd2-e13e-4e12-8e3f-387c4b18e525 None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:00.886539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589ddd2-e13e-4e12-8e3f-387c4b18e525 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:00.886691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-b41419d2-0edf-4f0a-93ef-5a817a60a12e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:05:00.886691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:00.886691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:00.886691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:00.886691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:00.886691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:00.886691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:00.886691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:00.886691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:00.886691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:00.886691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:00.886691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:00.886691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:05:00.886691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:05:00.886691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:05:00.886691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:00.887584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:00.888777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:00.888777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:00.888777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:05:00.888777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:05:00.888777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:05:00.888777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:05:00.888777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:05:00.888777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:05:00.888777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:05:00.888777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:00.888777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 345/697] 10.222.0.45 () {56 vars in 1093 bytes} [Thu Aug 4 02:05:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:00.890098 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589ddd2-e13e-4e12-8e3f-387c4b18e525 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:00.890282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41419d2-0edf-4f0a-93ef-5a817a60a12e None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:00.898108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589ddd2-e13e-4e12-8e3f-387c4b18e525 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:00.921221 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 353/698] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:05:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:00.922390 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589ddd2-e13e-4e12-8e3f-387c4b18e525 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:00.997529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41419d2-0edf-4f0a-93ef-5a817a60a12e None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:01.033356 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045702a6-b323-4a5e-ba55-6aab4e37ebf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:01.034760 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045702a6-b323-4a5e-ba55-6aab4e37ebf0 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:01.035136 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045702a6-b323-4a5e-ba55-6aab4e37ebf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:01.035529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045702a6-b323-4a5e-ba55-6aab4e37ebf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:01.047249 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045702a6-b323-4a5e-ba55-6aab4e37ebf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:01.048909 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045702a6-b323-4a5e-ba55-6aab4e37ebf0 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:01.052482 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:01.064787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-045702a6-b323-4a5e-ba55-6aab4e37ebf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:01.064787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:01.064787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:01.064787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:01.064787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:01.064787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:01.064787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:01.064787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:01.064787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:01.064787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:01.064787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:01.064787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:01.064787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:01.064787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:01.064787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:01.065881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:01.065881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:01.065881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:01.065881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:01.065881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:01.065881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:01.065881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:01.065881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:01.065881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:01.065881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:01.065881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:01.065881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:01.065881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:01.065881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:01.065881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:01.065881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:01.065881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:01.066971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:01.066971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:01.066971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:01.066971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:01.073397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 346/699] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:01.073397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045702a6-b323-4a5e-ba55-6aab4e37ebf0 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:01.148092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589ddd2-e13e-4e12-8e3f-387c4b18e525 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:01.210588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045702a6-b323-4a5e-ba55-6aab4e37ebf0 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:01.213407 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c54e90-2510-4cd0-88b0-e7ac9cd3d3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:01.214048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c54e90-2510-4cd0-88b0-e7ac9cd3d3f2 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:01.214323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c54e90-2510-4cd0-88b0-e7ac9cd3d3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:01.214586 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c54e90-2510-4cd0-88b0-e7ac9cd3d3f2 None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:01.215392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c54e90-2510-4cd0-88b0-e7ac9cd3d3f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:01.215694 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c54e90-2510-4cd0-88b0-e7ac9cd3d3f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:01.217149 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c54e90-2510-4cd0-88b0-e7ac9cd3d3f2 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:01.247337 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fa583-e6a4-4a49-9ea1-4c1c650e1e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:01.247965 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fa583-e6a4-4a49-9ea1-4c1c650e1e0a None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:01.248245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fa583-e6a4-4a49-9ea1-4c1c650e1e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:01.248520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fa583-e6a4-4a49-9ea1-4c1c650e1e0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:01.249540 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1fa583-e6a4-4a49-9ea1-4c1c650e1e0a None admin] RBAC: Adding query filter params (role.id=977824a8c6a24c00af2f6dabd0e2a6f0, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=155a7276efab426cb34175683c475f33) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:01.249859 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1fa583-e6a4-4a49-9ea1-4c1c650e1e0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:01.251308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1fa583-e6a4-4a49-9ea1-4c1c650e1e0a None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:01.265157 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 347/700] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:05:01 2022] GET /identity/v3/role_assignments?user.id=155a7276efab426cb34175683c475f33&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=977824a8c6a24c00af2f6dabd0e2a6f0 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:01.266213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fa583-e6a4-4a49-9ea1-4c1c650e1e0a None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:01.271797 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 354/701] 10.222.0.45 () {56 vars in 1133 bytes} [Thu Aug 4 02:05:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:01.277206 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c54e90-2510-4cd0-88b0-e7ac9cd3d3f2 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:01.356659 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:01.446841 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:01.465394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c54e90-2510-4cd0-88b0-e7ac9cd3d3f2 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:01.505597 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d72ec4-96f7-48f5-9402-c24173512fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:01.506503 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d72ec4-96f7-48f5-9402-c24173512fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:01.506929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d72ec4-96f7-48f5-9402-c24173512fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:01.507347 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d72ec4-96f7-48f5-9402-c24173512fb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:01.522249 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fa583-e6a4-4a49-9ea1-4c1c650e1e0a None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:01.558105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fcacb-be5e-4ba5-95f9-31572ac72351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:01.558765 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fcacb-be5e-4ba5-95f9-31572ac72351 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:01.559003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fcacb-be5e-4ba5-95f9-31572ac72351 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:01.559284 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6fcacb-be5e-4ba5-95f9-31572ac72351 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:01.559983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fcacb-be5e-4ba5-95f9-31572ac72351 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:01.560291 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fcacb-be5e-4ba5-95f9-31572ac72351 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:01.561724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6fcacb-be5e-4ba5-95f9-31572ac72351 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:01.576657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 348/702] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:01.586994 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d72ec4-96f7-48f5-9402-c24173512fb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:01.588451 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d72ec4-96f7-48f5-9402-c24173512fb1 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:01.600068 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fcacb-be5e-4ba5-95f9-31572ac72351 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:01.627340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:01.729322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 355/703] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:05:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 04 02:05:01.730715 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af6e8c-b3c9-47a4-bd3f-8f906fd5864d None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:01.731002 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af6e8c-b3c9-47a4-bd3f-8f906fd5864d None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:01.731280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af6e8c-b3c9-47a4-bd3f-8f906fd5864d None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:01.732840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 356/704] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:01.740581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af6e8c-b3c9-47a4-bd3f-8f906fd5864d None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:01.745796 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6fcacb-be5e-4ba5-95f9-31572ac72351 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:01.766812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:01.792230 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773757f-5701-4736-8d66-2586c680ef35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:01.793344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773757f-5701-4736-8d66-2586c680ef35 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:01.793637 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773757f-5701-4736-8d66-2586c680ef35 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:01.793914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773757f-5701-4736-8d66-2586c680ef35 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:01.794598 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773757f-5701-4736-8d66-2586c680ef35 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:01.796784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1773757f-5701-4736-8d66-2586c680ef35 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:01.827558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 349/705] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:05:01 2022] POST /identity/v3/users => generated 254 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:05:01.828974 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40314f0d-ac26-4149-9c59-a2df1969a7f8 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:01.829255 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40314f0d-ac26-4149-9c59-a2df1969a7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:01.829530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40314f0d-ac26-4149-9c59-a2df1969a7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:01.876133 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-40314f0d-ac26-4149-9c59-a2df1969a7f8 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:01.881589 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-40314f0d-ac26-4149-9c59-a2df1969a7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:01.898876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af6e8c-b3c9-47a4-bd3f-8f906fd5864d None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:01.948134 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:01.979113 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a9823-19c1-402f-befd-2f3882989af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:01.980223 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a9823-19c1-402f-befd-2f3882989af7 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:01.980741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a9823-19c1-402f-befd-2f3882989af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:01.981333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a9823-19c1-402f-befd-2f3882989af7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:02.005990 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a9823-19c1-402f-befd-2f3882989af7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:02.006357 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 350/706] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:05:02.008103 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41a9823-19c1-402f-befd-2f3882989af7 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:02.033750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-b41a9823-19c1-402f-befd-2f3882989af7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:02.033750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:02.033750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:02.033750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:02.033750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:02.033750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:02.033750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:02.033750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:02.033750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:02.033750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:02.033750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:02.033750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:02.033750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:02.033750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:02.033750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:02.035320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:02.035320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:02.035320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:02.035320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:02.035320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:02.035320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:02.035320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:02.035320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:02.035320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:02.035320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:02.035320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:02.035320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:02.035320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:02.035320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:02.035320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:02.035320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:02.035320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:02.036244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:02.036244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:02.036244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:02.036244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:02.038718 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 357/707] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:02.044126 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40314f0d-ac26-4149-9c59-a2df1969a7f8 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:02.045495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a9823-19c1-402f-befd-2f3882989af7 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:02.164093 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40314f0d-ac26-4149-9c59-a2df1969a7f8 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:02.166634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:02.176653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a9823-19c1-402f-befd-2f3882989af7 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:02.205490 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d68e84-819b-48bd-9bb3-c6e2f15f761b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:02.206767 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d68e84-819b-48bd-9bb3-c6e2f15f761b None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:02.207046 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d68e84-819b-48bd-9bb3-c6e2f15f761b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:02.207320 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d68e84-819b-48bd-9bb3-c6e2f15f761b None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:02.208132 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d68e84-819b-48bd-9bb3-c6e2f15f761b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:02.208437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d68e84-819b-48bd-9bb3-c6e2f15f761b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:02.209960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d68e84-819b-48bd-9bb3-c6e2f15f761b None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:02.217763 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6c70a-0475-47fb-98b7-f37c228f92c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:02.218835 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6c70a-0475-47fb-98b7-f37c228f92c9 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:02.219319 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6c70a-0475-47fb-98b7-f37c228f92c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:02.219865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6c70a-0475-47fb-98b7-f37c228f92c9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:02.220770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6c70a-0475-47fb-98b7-f37c228f92c9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:02.222297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6c70a-0475-47fb-98b7-f37c228f92c9 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:02.223901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 351/708] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:02.225151 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c7ad6-47fa-46e1-b009-a4cd4e4a76d2 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:02.229832 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c7ad6-47fa-46e1-b009-a4cd4e4a76d2 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:02.229832 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c7ad6-47fa-46e1-b009-a4cd4e4a76d2 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:02.229832 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 352/709] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:02.233661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c7ad6-47fa-46e1-b009-a4cd4e4a76d2 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:02.236369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-51d6c70a-0475-47fb-98b7-f37c228f92c9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 02:05:02.236369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:02.236369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:02.236369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:02.236369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:02.236369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:02.236369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:02.236369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:02.236369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 02:05:02.236369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:02.236369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:02.236369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:02.236369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:02.236369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:02.236369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:02.236369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:02.236369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:02.237511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:02.237511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:02.237511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:02.237511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:02.237511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:02.237511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:02.237511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:02.237511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:02.237511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:02.237511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:02.237511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:02.237511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:02.237511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:02.237511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:02.237511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:02.237511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:02.238359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:02.238359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:02.238359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:02.238359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:02.238359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:02.238359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:02.238359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:02.238359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:02.238359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:02.238359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:02.238359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:02.238359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:02.238359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 02:05:02.238359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:02.238359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 358/710] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:02 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:02.239660 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b29eed-a2b8-4a02-9aef-29fdc2bef823 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:02.239660 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b29eed-a2b8-4a02-9aef-29fdc2bef823 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:02.239813 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b29eed-a2b8-4a02-9aef-29fdc2bef823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:02.258521 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-34b29eed-a2b8-4a02-9aef-29fdc2bef823 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:02.259732 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-34b29eed-a2b8-4a02-9aef-29fdc2bef823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:02.292709 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:02.333086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 359/711] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:05:02.333765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b29eed-a2b8-4a02-9aef-29fdc2bef823 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:02.358591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c7ad6-47fa-46e1-b009-a4cd4e4a76d2 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:02.398815 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f4fcf-4f3c-4ac8-968f-698f574039c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:02.399849 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f4fcf-4f3c-4ac8-968f-698f574039c9 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:02.400341 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f4fcf-4f3c-4ac8-968f-698f574039c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:02.400840 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f4fcf-4f3c-4ac8-968f-698f574039c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:02.408252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f4fcf-4f3c-4ac8-968f-698f574039c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:02.410592 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f4fcf-4f3c-4ac8-968f-698f574039c9 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:02.426838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-748f4fcf-4f3c-4ac8-968f-698f574039c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:02.426838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:02.426838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:02.426838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:02.426838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:02.426838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:02.426838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:02.426838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:02.426838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:02.426838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:02.426838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:02.426838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:02.426838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:05:02.426838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:05:02.426838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:05:02.427784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:05:02.427784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:02.427784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:02.427784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:02.427784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:02.427784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:02.427784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:02.427784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:02.427784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:02.427784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:02.427784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:02.427784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:02.427784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:02.427784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:02.427784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:02.427784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:02.427784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:02.428715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:05:02.428715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:05:02.428715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:05:02.428715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:05:02.428715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:05:02.428715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:05:02.428715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:02.428715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:02.436323 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 353/712] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:02.436323 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f4fcf-4f3c-4ac8-968f-698f574039c9 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:02.444390 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b29eed-a2b8-4a02-9aef-29fdc2bef823 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:02.473026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:02.482140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288eb75f-8bb7-48c0-bba7-cda199859ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:02.482352 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eb75f-8bb7-48c0-bba7-cda199859ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:02.482553 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eb75f-8bb7-48c0-bba7-cda199859ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:02.482768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eb75f-8bb7-48c0-bba7-cda199859ed7 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:02.483439 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288eb75f-8bb7-48c0-bba7-cda199859ed7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:02.483788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288eb75f-8bb7-48c0-bba7-cda199859ed7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:02.485238 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288eb75f-8bb7-48c0-bba7-cda199859ed7 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:02.495016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 360/713] 10.222.0.45 () {56 vars in 1112 bytes} [Thu Aug 4 02:05:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:02.495016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288eb75f-8bb7-48c0-bba7-cda199859ed7 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:02.549037 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:02.560590 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f4fcf-4f3c-4ac8-968f-698f574039c9 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:02.597285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288eb75f-8bb7-48c0-bba7-cda199859ed7 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:02.601536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24310b25-ca58-4d56-8c4f-8387ba552807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:02.601684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24310b25-ca58-4d56-8c4f-8387ba552807 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:02.602083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24310b25-ca58-4d56-8c4f-8387ba552807 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:02.602083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24310b25-ca58-4d56-8c4f-8387ba552807 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:02.603875 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24310b25-ca58-4d56-8c4f-8387ba552807 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:02.605641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24310b25-ca58-4d56-8c4f-8387ba552807 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:02.619523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-24310b25-ca58-4d56-8c4f-8387ba552807 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 02:05:02.619523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:02.619523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:02.619523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:02.619523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:02.619523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:02.619523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:02.619523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:02.619523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 02:05:02.619523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:02.619523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:02.619523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:02.619523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:02.619523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:02.619523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:02.619523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:02.620551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:02.620551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:02.620551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:02.620551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:02.620551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:02.620551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:02.620551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:02.620551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:02.620551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:02.620551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:02.620551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:02.620551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:02.620551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:02.620551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:02.620551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:02.620551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:02.621474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:02.621474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:02.621474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:02.621474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:02.621474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:02.621474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:02.621474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:02.621474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:02.621474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:02.621474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:02.621474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:02.621474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:02.621474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:02.621474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 02:05:02.621474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:02.621474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 354/714] 10.222.0.45 () {56 vars in 1129 bytes} [Thu Aug 4 02:05:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:02.625175 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24310b25-ca58-4d56-8c4f-8387ba552807 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:02.634368 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a88c8c-8634-48d7-ae06-4749f9085698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:02.634981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a88c8c-8634-48d7-ae06-4749f9085698 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:02.635255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a88c8c-8634-48d7-ae06-4749f9085698 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:02.635528 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a88c8c-8634-48d7-ae06-4749f9085698 None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:02.636271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a88c8c-8634-48d7-ae06-4749f9085698 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:02.636564 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a88c8c-8634-48d7-ae06-4749f9085698 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:02.637985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a88c8c-8634-48d7-ae06-4749f9085698 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:02.650806 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 361/715] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:05:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:02.651648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a88c8c-8634-48d7-ae06-4749f9085698 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:02.728456 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:02.743975 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24310b25-ca58-4d56-8c4f-8387ba552807 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:02.748844 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:02.759890 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a88c8c-8634-48d7-ae06-4749f9085698 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:02.786352 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aaf2ac-5509-41df-a0e9-448be8476713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:02.787239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaf2ac-5509-41df-a0e9-448be8476713 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:02.787757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaf2ac-5509-41df-a0e9-448be8476713 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:02.788191 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aaf2ac-5509-41df-a0e9-448be8476713 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:02.789196 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aaf2ac-5509-41df-a0e9-448be8476713 None admin] RBAC: Adding query filter params (type=network) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:02.789645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aaf2ac-5509-41df-a0e9-448be8476713 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:02.791307 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aaf2ac-5509-41df-a0e9-448be8476713 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:02.796650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ea0b5-0a00-4a4c-a387-a452dce0b0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:02.797268 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ea0b5-0a00-4a4c-a387-a452dce0b0ee None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:02.797548 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ea0b5-0a00-4a4c-a387-a452dce0b0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:02.797817 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ea0b5-0a00-4a4c-a387-a452dce0b0ee None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:02.798569 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ea0b5-0a00-4a4c-a387-a452dce0b0ee None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:02.798752 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 355/716] 10.222.0.45 () {56 vars in 1112 bytes} [Thu Aug 4 02:05:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:02.798874 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ea0b5-0a00-4a4c-a387-a452dce0b0ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:02.800385 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aaf2ac-5509-41df-a0e9-448be8476713 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:02.800652 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ea0b5-0a00-4a4c-a387-a452dce0b0ee None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:02.806852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 362/717] 10.222.0.45 () {56 vars in 1124 bytes} [Thu Aug 4 02:05:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:02.815042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ea0b5-0a00-4a4c-a387-a452dce0b0ee None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:02.996380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ea0b5-0a00-4a4c-a387-a452dce0b0ee None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:03.011238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aaf2ac-5509-41df-a0e9-448be8476713 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:03.044279 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149664d-2a67-41a8-8a4c-4ac8b1f31b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:03.045379 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149664d-2a67-41a8-8a4c-4ac8b1f31b9d None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:03.045799 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149664d-2a67-41a8-8a4c-4ac8b1f31b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:03.046236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149664d-2a67-41a8-8a4c-4ac8b1f31b9d None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:03.047063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8149664d-2a67-41a8-8a4c-4ac8b1f31b9d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:03.047683 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8149664d-2a67-41a8-8a4c-4ac8b1f31b9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:03.048855 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0addd7ac-3ae6-4d59-a1da-23acd570ea3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:03.049422 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8149664d-2a67-41a8-8a4c-4ac8b1f31b9d None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:03.049720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0addd7ac-3ae6-4d59-a1da-23acd570ea3e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:03.050105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0addd7ac-3ae6-4d59-a1da-23acd570ea3e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:03.050479 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0addd7ac-3ae6-4d59-a1da-23acd570ea3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:03.051377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0addd7ac-3ae6-4d59-a1da-23acd570ea3e None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=c571406d2a3f40dab035a918399d6f6a) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:03.051842 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0addd7ac-3ae6-4d59-a1da-23acd570ea3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:03.053431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0addd7ac-3ae6-4d59-a1da-23acd570ea3e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:03.063029 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 363/719] 10.222.0.45 () {56 vars in 1124 bytes} [Thu Aug 4 02:05:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:03.068880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 356/719] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:05:02 2022] GET /identity/v3/role_assignments?user.id=c571406d2a3f40dab035a918399d6f6a&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=35205394c8af4c52a61b12f92ba16644 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:03.779108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eec95-65c0-4d78-9e35-c6685bbebc31 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:03.780155 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eec95-65c0-4d78-9e35-c6685bbebc31 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:03.780668 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6eec95-65c0-4d78-9e35-c6685bbebc31 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:03.782585 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 364/720] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:03.788538 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f228d-ef30-4e3f-ab96-72ab59ee9a10 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:03.789506 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f228d-ef30-4e3f-ab96-72ab59ee9a10 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:03.790029 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f228d-ef30-4e3f-ab96-72ab59ee9a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:03.817148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-8c5f228d-ef30-4e3f-ab96-72ab59ee9a10 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:03.818701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-8c5f228d-ef30-4e3f-ab96-72ab59ee9a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:03.831182 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:03.914375 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 357/721] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:05:03.954005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6eec95-65c0-4d78-9e35-c6685bbebc31 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:03.979670 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:04.220000 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6eec95-65c0-4d78-9e35-c6685bbebc31 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:04.270055 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4b4b9-abd9-4084-8d4f-c85d2a5d8ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:04.270704 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4b4b9-abd9-4084-8d4f-c85d2a5d8ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:04.270983 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4b4b9-abd9-4084-8d4f-c85d2a5d8ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:04.271257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4b4b9-abd9-4084-8d4f-c85d2a5d8ba3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:04.287461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4b4b9-abd9-4084-8d4f-c85d2a5d8ba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:05:04.292386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-0eb4b4b9-abd9-4084-8d4f-c85d2a5d8ba3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:05:04.292386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:04.292386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:04.292386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:04.292386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:04.292386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:04.292386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:04.292386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:04.292386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:04.292386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:04.292386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:04.292386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:04.292386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:05:04.292386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:05:04.292386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:05:04.292386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:05:04.293340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:05:04.293340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:05:04.293340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:05:04.293340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:05:04.293340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:04.293340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:04.293340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:04.293340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:04.293340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:04.293340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:04.293340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:04.293340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:04.293340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:04.293340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:04.293340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:04.293340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:04.294202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:04.294202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:04.294202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:04.294202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:04.294202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:05:04.294202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:05:04.294202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:05:04.294202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:05:04.294202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:05:04.294202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:05:04.294202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:05:04.294202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:04.294202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 365/722] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:05:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:04.302070 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f228d-ef30-4e3f-ab96-72ab59ee9a10 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:04.364199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76c335-e55a-4bd5-b08e-5cf128f6013a None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:04.364199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76c335-e55a-4bd5-b08e-5cf128f6013a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:04.364478 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b76c335-e55a-4bd5-b08e-5cf128f6013a None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:04.365953 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 366/723] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:05:04.371406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fc91f-2237-4ddb-a10f-99e04af1babd None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:04.371754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fc91f-2237-4ddb-a10f-99e04af1babd None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:04.371950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fc91f-2237-4ddb-a10f-99e04af1babd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:04.400062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-b96fc91f-2237-4ddb-a10f-99e04af1babd None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:04.402800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b96fc91f-2237-4ddb-a10f-99e04af1babd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:04.478807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 367/724] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:05:04.501651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fc91f-2237-4ddb-a10f-99e04af1babd None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:04.505535 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f228d-ef30-4e3f-ab96-72ab59ee9a10 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:04.521069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:04.573542 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ecc62-cab7-4596-b3c9-61c87384a1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:04.574163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ecc62-cab7-4596-b3c9-61c87384a1ae None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:04.574438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ecc62-cab7-4596-b3c9-61c87384a1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:04.574707 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ecc62-cab7-4596-b3c9-61c87384a1ae None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:04.575422 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ecc62-cab7-4596-b3c9-61c87384a1ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:04.575721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ecc62-cab7-4596-b3c9-61c87384a1ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:04.577594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ecc62-cab7-4596-b3c9-61c87384a1ae None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:04.588491 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 358/725] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:05:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:04.594274 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ecc62-cab7-4596-b3c9-61c87384a1ae None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:04.617006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:04.702299 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fc91f-2237-4ddb-a10f-99e04af1babd None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:04.770146 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1296b-b3b4-40c4-86cb-be1ed29133f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:04.770702 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1296b-b3b4-40c4-86cb-be1ed29133f1 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:04.771636 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1296b-b3b4-40c4-86cb-be1ed29133f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:04.771908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1296b-b3b4-40c4-86cb-be1ed29133f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:04.785164 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ecc62-cab7-4596-b3c9-61c87384a1ae None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:04.794580 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d1296b-b3b4-40c4-86cb-be1ed29133f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:05:04.802173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-a2d1296b-b3b4-40c4-86cb-be1ed29133f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:05:04.802173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:04.802173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:04.802173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:04.802173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:04.802173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:04.802173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:04.802173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:04.802173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:04.802173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:04.802173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:04.802173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:04.802173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:05:04.802173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:05:04.802173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:05:04.802173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:05:04.803182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:05:04.803182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:05:04.803182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:05:04.803182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:05:04.803182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:04.803182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:04.803182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:04.803182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:04.803182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:04.803182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:04.803182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:04.803182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:04.803182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:04.803182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:04.803182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:04.803182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:04.804022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:04.804022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:04.804022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:04.804022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:04.804022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:05:04.804022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:05:04.804022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:05:04.804022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:05:04.804022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:05:04.804022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:05:04.804022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:05:04.804022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:04.807753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 368/726] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:05:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:04.814116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1296b-b3b4-40c4-86cb-be1ed29133f1 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:04.854227 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472aa6b5-58ce-4db4-a235-a56286cec699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:04.855601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472aa6b5-58ce-4db4-a235-a56286cec699 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:04.856090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472aa6b5-58ce-4db4-a235-a56286cec699 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:04.856567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472aa6b5-58ce-4db4-a235-a56286cec699 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:04.873795 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472aa6b5-58ce-4db4-a235-a56286cec699 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:04.876639 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472aa6b5-58ce-4db4-a235-a56286cec699 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:04.904940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-472aa6b5-58ce-4db4-a235-a56286cec699 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:04.904940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:04.904940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:04.904940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:04.904940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:04.904940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:04.904940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:04.904940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:04.904940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:04.904940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:04.904940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:04.904940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:04.904940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:04.904940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:04.904940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:04.905899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:04.905899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:04.905899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:04.905899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:04.905899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:04.905899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:04.905899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:04.905899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:04.905899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:04.905899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:04.905899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:04.905899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:04.905899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:04.905899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:04.905899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:04.905899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:04.905899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:04.906796 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:04.906796 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:04.906796 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:04.906796 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:04.913199 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 359/727] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:04.925180 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472aa6b5-58ce-4db4-a235-a56286cec699 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:04.962612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:04.989119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:05.032052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472aa6b5-58ce-4db4-a235-a56286cec699 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:05.067345 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1296b-b3b4-40c4-86cb-be1ed29133f1 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:05.069722 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b141d-480c-4843-bfb8-4cf2ff1996a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:05.070368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b141d-480c-4843-bfb8-4cf2ff1996a8 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:05.070629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b141d-480c-4843-bfb8-4cf2ff1996a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:05.070904 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b141d-480c-4843-bfb8-4cf2ff1996a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:05.071595 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b141d-480c-4843-bfb8-4cf2ff1996a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:05.071897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b141d-480c-4843-bfb8-4cf2ff1996a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:05.073411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b141d-480c-4843-bfb8-4cf2ff1996a8 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:05.086728 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 360/728] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:05.092080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b141d-480c-4843-bfb8-4cf2ff1996a8 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:05.141372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcfc23-adb7-4fee-8bd6-85dad055cee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:05.142027 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcfc23-adb7-4fee-8bd6-85dad055cee9 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:05.142314 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcfc23-adb7-4fee-8bd6-85dad055cee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:05.142590 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcfc23-adb7-4fee-8bd6-85dad055cee9 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:05.143398 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcfc23-adb7-4fee-8bd6-85dad055cee9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:05.144199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcfc23-adb7-4fee-8bd6-85dad055cee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:05.145602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bcfc23-adb7-4fee-8bd6-85dad055cee9 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:05.175033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 369/729] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:05:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:05.193646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcfc23-adb7-4fee-8bd6-85dad055cee9 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:05.200050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b141d-480c-4843-bfb8-4cf2ff1996a8 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:05.225038 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:05.240310 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7c111-fadb-4d18-a9cf-91c517635922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:05.241077 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7c111-fadb-4d18-a9cf-91c517635922 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:05.241357 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7c111-fadb-4d18-a9cf-91c517635922 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:05.241626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c7c111-fadb-4d18-a9cf-91c517635922 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:05.250046 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7c111-fadb-4d18-a9cf-91c517635922 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:05.251513 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c7c111-fadb-4d18-a9cf-91c517635922 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:05.264799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-91c7c111-fadb-4d18-a9cf-91c517635922 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 02:05:05.264799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:05.264799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:05.264799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:05.264799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:05.264799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:05.264799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:05.264799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:05.264799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:05.264799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:05.264799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:05.264799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:05.264799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:05:05.264799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:05:05.264799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:05:05.264799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:05.267089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:05.272267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:05.272267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:05.272267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:05:05.272267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:05:05.272267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:05:05.272267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:05:05.272267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:05:05.272267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:05:05.272267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 02:05:05.272267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:05.272267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 361/730] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:05:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:05.273964 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7c111-fadb-4d18-a9cf-91c517635922 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:05.386503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c7c111-fadb-4d18-a9cf-91c517635922 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:05.423893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd19b6-9af9-4691-873c-db74d6d0a359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:05.425245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd19b6-9af9-4691-873c-db74d6d0a359 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:05.425436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd19b6-9af9-4691-873c-db74d6d0a359 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:05.425717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd19b6-9af9-4691-873c-db74d6d0a359 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:05.428920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd19b6-9af9-4691-873c-db74d6d0a359 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:05.429441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd19b6-9af9-4691-873c-db74d6d0a359 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:05.431099 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd19b6-9af9-4691-873c-db74d6d0a359 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:05.468412 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 362/731] 10.222.0.45 () {56 vars in 1135 bytes} [Thu Aug 4 02:05:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:05.473663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd19b6-9af9-4691-873c-db74d6d0a359 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:05.482529 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bcfc23-adb7-4fee-8bd6-85dad055cee9 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:05.516087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:05.517150 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:05.525251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc13ab-af90-4306-83d3-95f767625cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:05.525887 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc13ab-af90-4306-83d3-95f767625cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:05.526161 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc13ab-af90-4306-83d3-95f767625cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:05.526433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc13ab-af90-4306-83d3-95f767625cc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:05.538883 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc13ab-af90-4306-83d3-95f767625cc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:05.540308 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc13ab-af90-4306-83d3-95f767625cc1 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:05.554826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-7bdc13ab-af90-4306-83d3-95f767625cc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:05.554826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:05.554826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:05.554826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:05.554826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:05.554826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:05.554826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:05.554826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:05.554826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:05.554826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:05.554826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:05.554826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:05.554826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:05.554826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:05.554826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:05.555782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:05.555782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:05.555782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:05.555782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:05.555782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:05.555782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:05.555782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:05.555782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:05.555782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:05.555782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:05.555782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:05.555782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:05.555782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:05.555782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:05.555782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:05.555782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:05.555782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:05.556736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:05.556736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:05.556736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:05.556736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:05.556736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 370/732] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:05.557941 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552980b8-e2fb-40e7-b44c-dd1f2d621362 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:05.557941 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552980b8-e2fb-40e7-b44c-dd1f2d621362 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:05.558105 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552980b8-e2fb-40e7-b44c-dd1f2d621362 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:05.563453 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 371/733] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:05.563453 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abe4b3-1289-416f-b0d3-6e1c3de6844d None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:05.563453 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abe4b3-1289-416f-b0d3-6e1c3de6844d None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:05.563453 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abe4b3-1289-416f-b0d3-6e1c3de6844d None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:05.563453 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 372/734] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:05.563453 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abe4b3-1289-416f-b0d3-6e1c3de6844d None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:05.588765 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd19b6-9af9-4691-873c-db74d6d0a359 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:05.625680 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2901e-0f58-43f2-a77e-853b08b6a435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:05.626305 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2901e-0f58-43f2-a77e-853b08b6a435 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:05.626702 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2901e-0f58-43f2-a77e-853b08b6a435 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:05.626988 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc2901e-0f58-43f2-a77e-853b08b6a435 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:05.682061 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abe4b3-1289-416f-b0d3-6e1c3de6844d None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:05.700138 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2901e-0f58-43f2-a77e-853b08b6a435 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:05.701619 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc2901e-0f58-43f2-a77e-853b08b6a435 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:05.716257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8120aa4-0160-4234-9729-07efac72916d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:05.716924 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8120aa4-0160-4234-9729-07efac72916d None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:05.717202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8120aa4-0160-4234-9729-07efac72916d None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:05.717473 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8120aa4-0160-4234-9729-07efac72916d None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:05.718165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8120aa4-0160-4234-9729-07efac72916d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:05.718467 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8120aa4-0160-4234-9729-07efac72916d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:05.719815 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8120aa4-0160-4234-9729-07efac72916d None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:05.731933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 373/735] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:05.733220 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575b448-fb72-448c-b73d-8dce61210b7f None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:05.733493 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575b448-fb72-448c-b73d-8dce61210b7f None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:05.733767 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3575b448-fb72-448c-b73d-8dce61210b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:05.741061 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:05.751235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-3575b448-fb72-448c-b73d-8dce61210b7f None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:05.752418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-3575b448-fb72-448c-b73d-8dce61210b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:05.793664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:05.823016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 374/736] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:05:05.824508 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0bc69-f2fb-4437-8f2a-db50440cc358 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:05.824827 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0bc69-f2fb-4437-8f2a-db50440cc358 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:05.825101 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0bc69-f2fb-4437-8f2a-db50440cc358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:05.844461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-0cc0bc69-f2fb-4437-8f2a-db50440cc358 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:05.845888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-0cc0bc69-f2fb-4437-8f2a-db50440cc358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:05.917509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 375/737] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:05:05.918693 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc216f-bac8-4ef4-9624-1ef972ac4716 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:05.918973 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc216f-bac8-4ef4-9624-1ef972ac4716 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:05.919244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bc216f-bac8-4ef4-9624-1ef972ac4716 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:05.920477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 376/738] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:05.921205 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc216f-bac8-4ef4-9624-1ef972ac4716 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:05.932972 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 363/739] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:05:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 04 02:05:05.933641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2901e-0f58-43f2-a77e-853b08b6a435 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:06.025926 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bc216f-bac8-4ef4-9624-1ef972ac4716 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:06.028172 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:06.030961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc2901e-0f58-43f2-a77e-853b08b6a435 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:06.060082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5b1c1-17bd-4ed7-8a01-627e47b8a173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:06.060752 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5b1c1-17bd-4ed7-8a01-627e47b8a173 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:06.061030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5b1c1-17bd-4ed7-8a01-627e47b8a173 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:06.061304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5b1c1-17bd-4ed7-8a01-627e47b8a173 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:06.066053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac89a6-9678-45d5-ae63-baf20ed33ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:06.066663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac89a6-9678-45d5-ae63-baf20ed33ae9 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:06.066939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac89a6-9678-45d5-ae63-baf20ed33ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:06.067207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac89a6-9678-45d5-ae63-baf20ed33ae9 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:06.067886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac89a6-9678-45d5-ae63-baf20ed33ae9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:06.069273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac89a6-9678-45d5-ae63-baf20ed33ae9 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:06.069621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5b1c1-17bd-4ed7-8a01-627e47b8a173 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:06.071054 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5b1c1-17bd-4ed7-8a01-627e47b8a173 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:06.080217 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:06.085169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-4cc5b1c1-17bd-4ed7-8a01-627e47b8a173 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 02:05:06.085169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:06.085169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:06.085169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:06.085169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:06.085169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:06.085169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:06.085169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:06.085169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:06.085169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:06.085169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:06.085169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:06.085169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:05:06.085169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:05:06.085169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:05:06.085169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:06.087269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:06.089380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:06.089380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:06.089380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:05:06.089380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:05:06.089380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:05:06.089380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:05:06.089380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:05:06.089380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:05:06.089380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 02:05:06.089380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:06.089380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 377/740] 10.222.0.45 () {56 vars in 1117 bytes} [Thu Aug 4 02:05:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:06.089380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47663b-b0ad-4964-814f-2098558540fb None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:06.089380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47663b-b0ad-4964-814f-2098558540fb None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:06.090923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47663b-b0ad-4964-814f-2098558540fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:06.099247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 364/741] 10.222.0.45 () {60 vars in 1141 bytes} [Thu Aug 4 02:05:05 2022] POST /identity/v3/services => generated 240 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:05:06.099247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac89a6-9678-45d5-ae63-baf20ed33ae9 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:06.122837 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-4d47663b-b0ad-4964-814f-2098558540fb None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:06.124165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4d47663b-b0ad-4964-814f-2098558540fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:06.225072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 378/742] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:06 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 04 02:05:06.225072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47663b-b0ad-4964-814f-2098558540fb None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:06.229145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac89a6-9678-45d5-ae63-baf20ed33ae9 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:06.271525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f3a76-7b7e-4081-b4cc-6c52051cda74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:06.272853 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f3a76-7b7e-4081-b4cc-6c52051cda74 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:06.272853 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f3a76-7b7e-4081-b4cc-6c52051cda74 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:06.273086 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f3a76-7b7e-4081-b4cc-6c52051cda74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:06.285476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f3a76-7b7e-4081-b4cc-6c52051cda74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:06.287192 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f3a76-7b7e-4081-b4cc-6c52051cda74 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:06.302951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-598f3a76-7b7e-4081-b4cc-6c52051cda74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:06.302951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:06.302951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:06.302951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:06.302951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:06.302951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:06.302951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:06.302951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:06.302951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:06.302951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:06.302951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:06.302951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:06.302951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:06.302951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:06.302951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:06.304092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:06.304092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:06.304092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:06.304092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:06.304092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:06.304092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:06.304092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:06.304092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:06.304092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:06.304092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:06.304092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:06.304092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:06.304092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:06.304092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:06.304092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:06.304092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:06.304092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:06.305345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:06.305345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:06.305345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:06.305345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:06.305345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 365/743] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:06.309979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f3a76-7b7e-4081-b4cc-6c52051cda74 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:06.337166 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:06.347816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47663b-b0ad-4964-814f-2098558540fb None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:06.358599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:06.386351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9087e-be00-442d-8d5d-ee53ef28d3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:06.386974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9087e-be00-442d-8d5d-ee53ef28d3cc None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:06.387255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9087e-be00-442d-8d5d-ee53ef28d3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:06.387525 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9087e-be00-442d-8d5d-ee53ef28d3cc None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:06.388209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9087e-be00-442d-8d5d-ee53ef28d3cc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:06.389716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9087e-be00-442d-8d5d-ee53ef28d3cc None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:06.406792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 379/744] 10.222.0.45 () {60 vars in 1141 bytes} [Thu Aug 4 02:05:06 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:05:06.407660 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9087e-be00-442d-8d5d-ee53ef28d3cc None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:06.416683 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f3a76-7b7e-4081-b4cc-6c52051cda74 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:06.461654 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24886408-8d45-4261-86a7-4d274ce0a681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:06.462549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24886408-8d45-4261-86a7-4d274ce0a681 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:06.462979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24886408-8d45-4261-86a7-4d274ce0a681 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:06.463398 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24886408-8d45-4261-86a7-4d274ce0a681 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:06.464517 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24886408-8d45-4261-86a7-4d274ce0a681 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:06.465022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24886408-8d45-4261-86a7-4d274ce0a681 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:06.466788 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24886408-8d45-4261-86a7-4d274ce0a681 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:06.488434 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 366/745] 10.222.0.45 () {56 vars in 1149 bytes} [Thu Aug 4 02:05:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:06.489627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24886408-8d45-4261-86a7-4d274ce0a681 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:06.537898 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9087e-be00-442d-8d5d-ee53ef28d3cc None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:06.585031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049510ca-2aab-4c80-bd26-30cbcb2c4cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:06.585611 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049510ca-2aab-4c80-bd26-30cbcb2c4cef None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:06.585891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049510ca-2aab-4c80-bd26-30cbcb2c4cef None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:06.586262 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049510ca-2aab-4c80-bd26-30cbcb2c4cef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:06.601858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049510ca-2aab-4c80-bd26-30cbcb2c4cef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:05:06.608251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-049510ca-2aab-4c80-bd26-30cbcb2c4cef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:05:06.608251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:06.608251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:06.608251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:06.608251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:06.608251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:06.608251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:06.608251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:06.608251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:06.608251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:06.608251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:06.608251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:06.608251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:05:06.608251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:05:06.608251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:05:06.608251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:05:06.609479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:05:06.609479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:05:06.609479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:05:06.609479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:05:06.609479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:06.609479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:06.609479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:06.609479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:06.609479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:06.609479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:06.609479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:06.609479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:06.609479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:06.609479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:06.609479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:06.609479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:06.610365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:06.610365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:06.610365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:06.610365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:06.610365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:05:06.610365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:05:06.610365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:05:06.610365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:05:06.610365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:05:06.610365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:05:06.610365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:05:06.610365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:06.610365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 380/746] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:05:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:06.610365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049510ca-2aab-4c80-bd26-30cbcb2c4cef None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:06.625236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:06.627174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:06.631704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24886408-8d45-4261-86a7-4d274ce0a681 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:06.677952 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f3c9b-fb9f-400d-8f73-180141680c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:06.679426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f3c9b-fb9f-400d-8f73-180141680c08 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:06.680127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f3c9b-fb9f-400d-8f73-180141680c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:06.680819 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f3c9b-fb9f-400d-8f73-180141680c08 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:06.681813 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f3c9b-fb9f-400d-8f73-180141680c08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:06.682347 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f3c9b-fb9f-400d-8f73-180141680c08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:06.683953 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483f3c9b-fb9f-400d-8f73-180141680c08 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:06.697365 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 367/747] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:06.698848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f3c9b-fb9f-400d-8f73-180141680c08 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:06.738099 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049510ca-2aab-4c80-bd26-30cbcb2c4cef None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:06.772595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45230397-7afe-4c2f-9cfb-d43676350a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:06.773174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45230397-7afe-4c2f-9cfb-d43676350a2c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:06.773504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45230397-7afe-4c2f-9cfb-d43676350a2c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:06.773845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45230397-7afe-4c2f-9cfb-d43676350a2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:06.817026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f3c9b-fb9f-400d-8f73-180141680c08 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:06.851408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45230397-7afe-4c2f-9cfb-d43676350a2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:06.852953 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45230397-7afe-4c2f-9cfb-d43676350a2c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:06.860972 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9f0f5-0196-4abb-a465-2389d3e7166e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:06.861867 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9f0f5-0196-4abb-a465-2389d3e7166e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:06.862960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9f0f5-0196-4abb-a465-2389d3e7166e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:06.863489 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9f0f5-0196-4abb-a465-2389d3e7166e None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:06.864471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9f0f5-0196-4abb-a465-2389d3e7166e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:06.864971 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:06.866213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9f0f5-0196-4abb-a465-2389d3e7166e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:06.867868 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9f0f5-0196-4abb-a465-2389d3e7166e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:06.883685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 368/748] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:05:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:06.885144 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9f0f5-0196-4abb-a465-2389d3e7166e None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:06.912878 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:06.958251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 381/749] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:05:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:05:06.959108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45230397-7afe-4c2f-9cfb-d43676350a2c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:06.997805 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9f0f5-0196-4abb-a465-2389d3e7166e None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:07.037481 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4db3a-f6fc-4ac9-8533-43bab94015de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:07.038507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4db3a-f6fc-4ac9-8533-43bab94015de None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:07.039001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4db3a-f6fc-4ac9-8533-43bab94015de None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:07.039513 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a4db3a-f6fc-4ac9-8533-43bab94015de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:07.048373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a4db3a-f6fc-4ac9-8533-43bab94015de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:07.050990 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a4db3a-f6fc-4ac9-8533-43bab94015de None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:07.063898 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45230397-7afe-4c2f-9cfb-d43676350a2c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:07.064998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-83a4db3a-f6fc-4ac9-8533-43bab94015de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:07.064998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:07.064998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:07.064998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:07.064998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:07.064998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:07.064998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:07.064998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:07.064998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:07.064998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:07.064998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:07.064998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:07.064998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:05:07.064998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:05:07.064998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:05:07.065905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:05:07.065905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:07.065905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:07.065905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:07.065905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:07.065905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:07.065905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:07.065905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:07.065905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:07.065905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:07.065905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:07.065905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:07.065905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:07.065905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:07.065905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:07.065905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:07.065905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:07.067362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:05:07.067362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:05:07.067362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:05:07.067362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:05:07.067362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:05:07.067362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:05:07.067362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:07.067362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:07.067362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 369/750] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:07.067362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4db3a-f6fc-4ac9-8533-43bab94015de None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:07.082878 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:07.119290 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3c9c3-7619-4865-bb19-e58965efa685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:07.120391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3c9c3-7619-4865-bb19-e58965efa685 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:07.121090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3c9c3-7619-4865-bb19-e58965efa685 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:07.121588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3c9c3-7619-4865-bb19-e58965efa685 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:07.136295 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3c9c3-7619-4865-bb19-e58965efa685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:07.138374 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3c9c3-7619-4865-bb19-e58965efa685 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:07.154591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:07.157880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-d6c3c9c3-7619-4865-bb19-e58965efa685 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:07.157880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:07.157880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:07.157880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:07.157880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:07.157880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:07.157880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:07.157880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:07.157880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:07.157880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:07.157880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:07.157880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:07.157880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:07.157880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:07.157880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:07.158988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:07.158988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:07.158988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:07.158988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:07.158988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:07.158988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:07.158988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:07.158988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:07.158988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:07.158988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:07.158988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:07.158988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:07.158988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:07.158988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:07.158988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:07.158988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:07.158988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:07.159911 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:07.159911 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:07.159911 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:07.159911 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:07.164705 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 382/751] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:07.166715 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3c9c3-7619-4865-bb19-e58965efa685 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:07.184176 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a4db3a-f6fc-4ac9-8533-43bab94015de None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:07.220102 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e748e0-f562-4e33-b62c-a26d58ac20d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:07.220776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e748e0-f562-4e33-b62c-a26d58ac20d6 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:07.221056 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e748e0-f562-4e33-b62c-a26d58ac20d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:07.221326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e748e0-f562-4e33-b62c-a26d58ac20d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:07.233088 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e748e0-f562-4e33-b62c-a26d58ac20d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:07.234545 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e748e0-f562-4e33-b62c-a26d58ac20d6 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:07.249617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-a5e748e0-f562-4e33-b62c-a26d58ac20d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:07.249617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:07.249617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:07.249617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:07.249617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:07.249617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:07.249617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:07.249617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:07.249617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:07.249617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:07.249617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:07.249617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:07.249617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:07.249617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:07.249617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:07.250539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:07.250539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:07.250539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:07.250539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:07.250539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:07.250539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:07.250539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:07.250539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:07.250539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:07.250539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:07.250539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:07.250539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:07.250539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:07.250539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:07.250539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:07.250539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:07.250539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:07.251413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:07.251413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:07.251413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:07.251413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:07.251413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 370/752] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:07.252111 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e748e0-f562-4e33-b62c-a26d58ac20d6 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:07.297157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3c9c3-7619-4865-bb19-e58965efa685 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:07.334276 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634b96b-676c-4e79-b74f-075e04b712f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:07.335263 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634b96b-676c-4e79-b74f-075e04b712f6 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:07.335756 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634b96b-676c-4e79-b74f-075e04b712f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:07.336236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634b96b-676c-4e79-b74f-075e04b712f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:07.337267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634b96b-676c-4e79-b74f-075e04b712f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:07.337779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634b96b-676c-4e79-b74f-075e04b712f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:07.339453 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634b96b-676c-4e79-b74f-075e04b712f6 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:07.353337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 383/753] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:05:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:07.354575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634b96b-676c-4e79-b74f-075e04b712f6 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:07.363623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e748e0-f562-4e33-b62c-a26d58ac20d6 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:07.380368 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:07.400274 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa0c20-03af-4769-a4b2-e86956061ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:07.400924 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa0c20-03af-4769-a4b2-e86956061ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:07.401237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa0c20-03af-4769-a4b2-e86956061ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:07.401748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa0c20-03af-4769-a4b2-e86956061ad9 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:07.402308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa0c20-03af-4769-a4b2-e86956061ad9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:07.402614 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa0c20-03af-4769-a4b2-e86956061ad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:07.403977 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fa0c20-03af-4769-a4b2-e86956061ad9 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:07.405537 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:07.423122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 371/754] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:07.424010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa0c20-03af-4769-a4b2-e86956061ad9 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:07.461591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634b96b-676c-4e79-b74f-075e04b712f6 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:07.497456 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3ab75-f066-4027-8f6d-723237fabd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:07.498408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3ab75-f066-4027-8f6d-723237fabd88 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:07.498909 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3ab75-f066-4027-8f6d-723237fabd88 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:07.499371 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba3ab75-f066-4027-8f6d-723237fabd88 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:07.500247 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3ab75-f066-4027-8f6d-723237fabd88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:07.500793 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3ab75-f066-4027-8f6d-723237fabd88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:07.502354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba3ab75-f066-4027-8f6d-723237fabd88 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:07.514573 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 384/755] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:07.519993 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3ab75-f066-4027-8f6d-723237fabd88 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:07.530224 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa0c20-03af-4769-a4b2-e86956061ad9 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:07.567778 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d27d2-45f0-477d-96ea-914677d9ca57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:07.568311 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d27d2-45f0-477d-96ea-914677d9ca57 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:07.568585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d27d2-45f0-477d-96ea-914677d9ca57 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:07.568902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d27d2-45f0-477d-96ea-914677d9ca57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:07.569777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d27d2-45f0-477d-96ea-914677d9ca57 None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=155a7276efab426cb34175683c475f33) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:07.570083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d27d2-45f0-477d-96ea-914677d9ca57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:07.571618 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d27d2-45f0-477d-96ea-914677d9ca57 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:07.577947 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 372/756] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:05:07 2022] GET /identity/v3/role_assignments?user.id=155a7276efab426cb34175683c475f33&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=35205394c8af4c52a61b12f92ba16644 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:07.578890 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d27d2-45f0-477d-96ea-914677d9ca57 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:07.610349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:07.661003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:07.670694 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba3ab75-f066-4027-8f6d-723237fabd88 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:07.710169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d27d2-45f0-477d-96ea-914677d9ca57 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:07.747268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d3c86-2790-45d1-b6fb-8dc4d474c4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:07.748173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d3c86-2790-45d1-b6fb-8dc4d474c4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:07.748595 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d3c86-2790-45d1-b6fb-8dc4d474c4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:07.749051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d3c86-2790-45d1-b6fb-8dc4d474c4f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:07.757011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d3c86-2790-45d1-b6fb-8dc4d474c4f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:07.758754 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d3c86-2790-45d1-b6fb-8dc4d474c4f0 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:07.762773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dafeff4-f914-4136-8928-e2f5aae58ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:07.763547 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dafeff4-f914-4136-8928-e2f5aae58ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:07.763976 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dafeff4-f914-4136-8928-e2f5aae58ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:07.764369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dafeff4-f914-4136-8928-e2f5aae58ab9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:07.773003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-0e5d3c86-2790-45d1-b6fb-8dc4d474c4f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:07.773003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:07.773003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:07.773003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:07.773003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:07.773003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:07.773003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:07.773003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:07.773003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:07.773003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:07.773003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:07.773003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:07.773003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:05:07.773003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:05:07.773003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:05:07.774174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:05:07.774174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:07.774174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:07.774174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:07.774174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:07.774174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:07.774174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:07.774174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:07.774174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:07.774174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:07.774174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:07.774174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:07.774174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:07.774174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:07.774174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:07.774174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:07.774174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:07.775250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:05:07.775250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:05:07.775250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:05:07.775250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:05:07.775250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:05:07.775250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:05:07.775250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:07.775250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:07.776064 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dafeff4-f914-4136-8928-e2f5aae58ab9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:07.777622 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 373/757] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:07.777915 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dafeff4-f914-4136-8928-e2f5aae58ab9 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:07.784659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d3c86-2790-45d1-b6fb-8dc4d474c4f0 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:07.797475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-5dafeff4-f914-4136-8928-e2f5aae58ab9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:05:07.797475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:07.797475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:07.797475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:07.797475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:07.797475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:07.797475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:07.797475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:07.797475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:07.797475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:07.797475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:07.797475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:07.797475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:05:07.797475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:05:07.797475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:05:07.797475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:07.798486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:07.799440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:07.799440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:07.799440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:05:07.799440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:05:07.799440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:05:07.799440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:05:07.799440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:05:07.799440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:05:07.799440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:05:07.799440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:07.802227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 385/758] 10.222.0.45 () {56 vars in 1093 bytes} [Thu Aug 4 02:05:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:07.809023 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dafeff4-f914-4136-8928-e2f5aae58ab9 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:07.916729 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dafeff4-f914-4136-8928-e2f5aae58ab9 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:07.921547 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d3c86-2790-45d1-b6fb-8dc4d474c4f0 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:07.954131 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:07.970023 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f6dd0-8445-4568-91f2-fcc80b18a8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:07.972506 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f6dd0-8445-4568-91f2-fcc80b18a8bb None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:07.972928 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f6dd0-8445-4568-91f2-fcc80b18a8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:07.974035 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f6dd0-8445-4568-91f2-fcc80b18a8bb None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:07.975559 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f6dd0-8445-4568-91f2-fcc80b18a8bb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:07.977618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f6dd0-8445-4568-91f2-fcc80b18a8bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:07.977618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f6dd0-8445-4568-91f2-fcc80b18a8bb None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:07.989713 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5321f-7aa2-4386-ba44-710da8b0cdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:07.990547 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5321f-7aa2-4386-ba44-710da8b0cdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:07.991749 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5321f-7aa2-4386-ba44-710da8b0cdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:07.992133 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b5321f-7aa2-4386-ba44-710da8b0cdf3 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:07.994803 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5321f-7aa2-4386-ba44-710da8b0cdf3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:07.995238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5321f-7aa2-4386-ba44-710da8b0cdf3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:07.997959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b5321f-7aa2-4386-ba44-710da8b0cdf3 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:08.012730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:08.017180 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 386/759] 10.222.0.45 () {56 vars in 1133 bytes} [Thu Aug 4 02:05:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:08.024366 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f6dd0-8445-4568-91f2-fcc80b18a8bb None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:08.030943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 374/760] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:05:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:08.041871 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5321f-7aa2-4386-ba44-710da8b0cdf3 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:08.196401 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b5321f-7aa2-4386-ba44-710da8b0cdf3 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:08.248210 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f6dd0-8445-4568-91f2-fcc80b18a8bb None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:08.258234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f77c3-dc93-4df5-93c9-c4aa733dca45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:08.259001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f77c3-dc93-4df5-93c9-c4aa733dca45 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:08.259394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f77c3-dc93-4df5-93c9-c4aa733dca45 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:08.259805 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f77c3-dc93-4df5-93c9-c4aa733dca45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:08.261190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f77c3-dc93-4df5-93c9-c4aa733dca45 None admin] RBAC: Adding query filter params (role.id=977824a8c6a24c00af2f6dabd0e2a6f0, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=e54e713b52e84393b018807f8153922e) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:08.261608 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f77c3-dc93-4df5-93c9-c4aa733dca45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:08.263850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f77c3-dc93-4df5-93c9-c4aa733dca45 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:08.271960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 375/761] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:05:08 2022] GET /identity/v3/role_assignments?user.id=e54e713b52e84393b018807f8153922e&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=977824a8c6a24c00af2f6dabd0e2a6f0 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:08.273608 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc671e-5783-4dfb-af2f-3e7cf16c4a78 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:08.274709 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc671e-5783-4dfb-af2f-3e7cf16c4a78 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:08.275122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc671e-5783-4dfb-af2f-3e7cf16c4a78 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:08.280727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 376/762] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:08.283008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d883dd-061f-4388-bc9b-20a6f81304e1 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:08.283291 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d883dd-061f-4388-bc9b-20a6f81304e1 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:08.283560 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d883dd-061f-4388-bc9b-20a6f81304e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:08.305465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-11d883dd-061f-4388-bc9b-20a6f81304e1 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:08.307935 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-11d883dd-061f-4388-bc9b-20a6f81304e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:08.318582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29f1a7-bdbc-429a-b1ef-3292ee23179f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:08.319636 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29f1a7-bdbc-429a-b1ef-3292ee23179f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:08.320055 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29f1a7-bdbc-429a-b1ef-3292ee23179f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:08.320476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e29f1a7-bdbc-429a-b1ef-3292ee23179f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:08.351156 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:08.352271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29f1a7-bdbc-429a-b1ef-3292ee23179f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:08.354098 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e29f1a7-bdbc-429a-b1ef-3292ee23179f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:08.387300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-9e29f1a7-bdbc-429a-b1ef-3292ee23179f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:08.387300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:08.387300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:08.387300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:08.387300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:08.387300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:08.387300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:08.387300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:08.387300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:08.387300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:08.387300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:08.387300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:08.387300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:08.387300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:08.387300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:08.388410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:08.388410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:08.388410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:08.388410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:08.388410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:08.388410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:08.388410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:08.388410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:08.388410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:08.388410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:08.388410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:08.388410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:08.388410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:08.388410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:08.388410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:08.388410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:08.388410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:08.389550 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:08.389550 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:08.389550 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:08.389550 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:08.396646 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 377/763] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:08.398639 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 387/764] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:08.400031 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d883dd-061f-4388-bc9b-20a6f81304e1 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:08.427970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29f1a7-bdbc-429a-b1ef-3292ee23179f None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:08.501894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:08.560529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d883dd-061f-4388-bc9b-20a6f81304e1 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:08.606090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e29f1a7-bdbc-429a-b1ef-3292ee23179f None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:08.623548 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd4dc8-40ad-4b20-8c0f-b60d8c394fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:08.624182 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd4dc8-40ad-4b20-8c0f-b60d8c394fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:08.624461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd4dc8-40ad-4b20-8c0f-b60d8c394fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:08.624788 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd4dc8-40ad-4b20-8c0f-b60d8c394fc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:08.625662 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd4dc8-40ad-4b20-8c0f-b60d8c394fc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:08.625967 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd4dc8-40ad-4b20-8c0f-b60d8c394fc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:08.628721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd4dc8-40ad-4b20-8c0f-b60d8c394fc7 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:08.644137 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 378/765] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:08.647200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817ea42-eda9-4763-93f5-a7f3e19a92b2 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:08.648559 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817ea42-eda9-4763-93f5-a7f3e19a92b2 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:08.649100 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817ea42-eda9-4763-93f5-a7f3e19a92b2 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:08.655891 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 379/766] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:08.657272 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df1354-3357-4365-99d2-1b3c5f56132a None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:08.657557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df1354-3357-4365-99d2-1b3c5f56132a None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:08.657830 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df1354-3357-4365-99d2-1b3c5f56132a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:08.672304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230bc55-befb-4b5f-8be3-407a9cd9afa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:08.673712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230bc55-befb-4b5f-8be3-407a9cd9afa3 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:08.674160 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230bc55-befb-4b5f-8be3-407a9cd9afa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:08.674615 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230bc55-befb-4b5f-8be3-407a9cd9afa3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:08.675650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230bc55-befb-4b5f-8be3-407a9cd9afa3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:08.677319 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230bc55-befb-4b5f-8be3-407a9cd9afa3 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:08.679013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-d1df1354-3357-4365-99d2-1b3c5f56132a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:08.680822 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d1df1354-3357-4365-99d2-1b3c5f56132a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:08.688515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-d230bc55-befb-4b5f-8be3-407a9cd9afa3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 02:05:08.688515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:08.688515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:08.688515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:08.688515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:08.688515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:08.688515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:08.688515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:08.688515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 02:05:08.688515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:08.688515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:08.688515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:08.688515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:08.688515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:08.688515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:08.688515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:08.688515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:08.689753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:08.689753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:08.689753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:08.689753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:08.689753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:08.689753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:08.689753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:08.689753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:08.689753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:08.689753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:08.689753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:08.689753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:08.689753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:08.689753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:08.689753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:08.689753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:08.690821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:08.690821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:08.690821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:08.690821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:08.690821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:08.690821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:08.690821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:08.690821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:08.690821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:08.690821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:08.690821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:08.690821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:08.690821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 02:05:08.690821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:08.693864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 388/767] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:08 2022] GET /identity/v3/services/network => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:08.695275 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230bc55-befb-4b5f-8be3-407a9cd9afa3 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:08.717117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:08.772668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 380/768] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:08.773762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df1354-3357-4365-99d2-1b3c5f56132a None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:08.832645 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230bc55-befb-4b5f-8be3-407a9cd9afa3 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:08.847021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:08.867518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39220e75-c2c2-4f05-8728-72f99722d2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:08.868310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39220e75-c2c2-4f05-8728-72f99722d2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:08.868764 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39220e75-c2c2-4f05-8728-72f99722d2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:08.869228 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39220e75-c2c2-4f05-8728-72f99722d2f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:08.876826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39220e75-c2c2-4f05-8728-72f99722d2f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:08.878455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39220e75-c2c2-4f05-8728-72f99722d2f0 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:08.891252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-39220e75-c2c2-4f05-8728-72f99722d2f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:08.891252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:08.891252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:08.891252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:08.891252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:08.891252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:08.891252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:08.891252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:08.891252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:08.891252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:08.891252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:08.891252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:08.891252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:05:08.891252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:05:08.891252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:05:08.892309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:05:08.892309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:08.892309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:08.892309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:08.892309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:08.892309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:08.892309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:08.892309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:08.892309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:08.892309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:08.892309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:08.892309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:08.892309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:08.892309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:08.892309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:08.892309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:08.892309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:08.893461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:05:08.893461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:05:08.893461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:05:08.893461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:05:08.893461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:05:08.893461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:05:08.893461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:08.893461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:08.893979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df1354-3357-4365-99d2-1b3c5f56132a None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:08.895983 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 389/769] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:08.899546 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39220e75-c2c2-4f05-8728-72f99722d2f0 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:08.930784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390bbb8-c2c4-40fa-a27a-e197b4c26e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:08.931567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390bbb8-c2c4-40fa-a27a-e197b4c26e18 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:08.931956 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390bbb8-c2c4-40fa-a27a-e197b4c26e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:08.932308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390bbb8-c2c4-40fa-a27a-e197b4c26e18 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:08.933161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390bbb8-c2c4-40fa-a27a-e197b4c26e18 None admin] RBAC: Adding query filter params (name=network) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:08.933576 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390bbb8-c2c4-40fa-a27a-e197b4c26e18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:08.935687 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390bbb8-c2c4-40fa-a27a-e197b4c26e18 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:08.942473 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 381/770] 10.222.0.45 () {56 vars in 1112 bytes} [Thu Aug 4 02:05:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:08.943406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390bbb8-c2c4-40fa-a27a-e197b4c26e18 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:08.972470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:09.006864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39220e75-c2c2-4f05-8728-72f99722d2f0 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:09.044508 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cd074-4fd0-4be3-83d2-392bde56ad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:09.045471 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cd074-4fd0-4be3-83d2-392bde56ad0b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:09.045921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cd074-4fd0-4be3-83d2-392bde56ad0b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:09.046340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cd074-4fd0-4be3-83d2-392bde56ad0b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:09.047120 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cd074-4fd0-4be3-83d2-392bde56ad0b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:09.048647 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cd074-4fd0-4be3-83d2-392bde56ad0b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:09.055778 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390bbb8-c2c4-40fa-a27a-e197b4c26e18 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:09.059505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-e25cd074-4fd0-4be3-83d2-392bde56ad0b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 02:05:09.059505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:09.059505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:09.059505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:09.059505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:09.059505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:09.059505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:09.059505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:09.059505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 02:05:09.059505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:09.059505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:09.059505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:09.059505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:09.059505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:09.059505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:09.059505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:09.060505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:09.060505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:09.060505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:09.060505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:09.060505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:09.060505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:09.060505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:09.060505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:09.060505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:09.060505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:09.060505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:09.060505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:09.060505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:09.060505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:09.060505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:09.060505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:09.061575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:09.061575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:09.061575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:09.061575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:09.061575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:09.061575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:09.061575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:09.061575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:09.061575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:09.061575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:09.061575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:09.061575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:09.061575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:09.061575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 02:05:09.061575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:09.069574 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 390/771] 10.222.0.45 () {56 vars in 1129 bytes} [Thu Aug 4 02:05:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:09.070678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cd074-4fd0-4be3-83d2-392bde56ad0b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:09.098303 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181dcc0-ca44-4c92-a1f6-9099bf242bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:09.099100 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181dcc0-ca44-4c92-a1f6-9099bf242bb9 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:09.099487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181dcc0-ca44-4c92-a1f6-9099bf242bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:09.099867 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9181dcc0-ca44-4c92-a1f6-9099bf242bb9 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:09.100869 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181dcc0-ca44-4c92-a1f6-9099bf242bb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:09.101520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181dcc0-ca44-4c92-a1f6-9099bf242bb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:09.102915 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9181dcc0-ca44-4c92-a1f6-9099bf242bb9 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:09.115567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:09.119223 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 382/772] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:05:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:09.120442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181dcc0-ca44-4c92-a1f6-9099bf242bb9 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:09.190682 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cd074-4fd0-4be3-83d2-392bde56ad0b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:09.229127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:09.235106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34254dd-cea4-4d00-9b7f-0b11ff225908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:09.235729 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34254dd-cea4-4d00-9b7f-0b11ff225908 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:09.236005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34254dd-cea4-4d00-9b7f-0b11ff225908 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:09.236280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34254dd-cea4-4d00-9b7f-0b11ff225908 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:09.236981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34254dd-cea4-4d00-9b7f-0b11ff225908 None admin] RBAC: Adding query filter params (type=network) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:09.237812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34254dd-cea4-4d00-9b7f-0b11ff225908 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:09.239233 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34254dd-cea4-4d00-9b7f-0b11ff225908 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:09.249304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 391/773] 10.222.0.45 () {56 vars in 1112 bytes} [Thu Aug 4 02:05:09 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:09.249304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34254dd-cea4-4d00-9b7f-0b11ff225908 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:09.261784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9181dcc0-ca44-4c92-a1f6-9099bf242bb9 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:09.308705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18496b12-77e4-4e63-8a23-e57acc9dfd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:09.310203 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18496b12-77e4-4e63-8a23-e57acc9dfd96 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:09.310671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18496b12-77e4-4e63-8a23-e57acc9dfd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:09.311379 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18496b12-77e4-4e63-8a23-e57acc9dfd96 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:09.312686 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18496b12-77e4-4e63-8a23-e57acc9dfd96 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:09.313071 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18496b12-77e4-4e63-8a23-e57acc9dfd96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:09.314459 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18496b12-77e4-4e63-8a23-e57acc9dfd96 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:09.324504 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 383/774] 10.222.0.45 () {56 vars in 1124 bytes} [Thu Aug 4 02:05:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:09.325393 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18496b12-77e4-4e63-8a23-e57acc9dfd96 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:09.367806 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34254dd-cea4-4d00-9b7f-0b11ff225908 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:09.371747 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:09.415998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e1765-f566-48b3-8ec4-b254f6b590ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:09.417755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e1765-f566-48b3-8ec4-b254f6b590ce None admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:09.417755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e1765-f566-48b3-8ec4-b254f6b590ce None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:09.417755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e1765-f566-48b3-8ec4-b254f6b590ce None admin] PATH_INFO: `/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/c571406d2a3f40dab035a918399d6f6a/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:09.419441 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e1765-f566-48b3-8ec4-b254f6b590ce None admin] RBAC: Authorizing `identity:create_grant(project_id=74b8caff828a4f5289094fb2ae2278f7, user_id=c571406d2a3f40dab035a918399d6f6a, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:09.420998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e1765-f566-48b3-8ec4-b254f6b590ce None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:09.441805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 392/775] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:05:09 2022] PUT /identity/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/c571406d2a3f40dab035a918399d6f6a/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:05:09.442833 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e1765-f566-48b3-8ec4-b254f6b590ce None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:09.452486 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18496b12-77e4-4e63-8a23-e57acc9dfd96 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:09.521692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f1bd6-1984-4593-bc3a-1bd93dd2118d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:09.522734 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f1bd6-1984-4593-bc3a-1bd93dd2118d None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:09.523343 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f1bd6-1984-4593-bc3a-1bd93dd2118d None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:09.523872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f1bd6-1984-4593-bc3a-1bd93dd2118d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:09.524959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f1bd6-1984-4593-bc3a-1bd93dd2118d None admin] RBAC: Adding query filter params (interface=public, service_id=c50be79a4c0e44ee86859f99ef8bc18f, region_id=RegionOne) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:09.525502 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f1bd6-1984-4593-bc3a-1bd93dd2118d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:09.527340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f1bd6-1984-4593-bc3a-1bd93dd2118d None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:09.547450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 384/776] 10.222.0.45 () {56 vars in 1251 bytes} [Thu Aug 4 02:05:09 2022] GET /identity/v3/endpoints?service_id=c50be79a4c0e44ee86859f99ef8bc18f&interface=public®ion_id=RegionOne => generated 194 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:09.553812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f1bd6-1984-4593-bc3a-1bd93dd2118d None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:09.580649 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e1765-f566-48b3-8ec4-b254f6b590ce None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:09.588284 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:09.628288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f7504-294e-4b73-80de-fdcd53bd4c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:09.629644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f7504-294e-4b73-80de-fdcd53bd4c15 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:09.630115 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f7504-294e-4b73-80de-fdcd53bd4c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:09.630595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f7504-294e-4b73-80de-fdcd53bd4c15 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:09.631764 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f7504-294e-4b73-80de-fdcd53bd4c15 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:09.632571 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f7504-294e-4b73-80de-fdcd53bd4c15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:09.634484 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f7504-294e-4b73-80de-fdcd53bd4c15 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:09.641539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 393/777] 10.222.0.45 () {56 vars in 1124 bytes} [Thu Aug 4 02:05:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:09.648236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f7504-294e-4b73-80de-fdcd53bd4c15 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:09.697119 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:09.708847 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f1bd6-1984-4593-bc3a-1bd93dd2118d None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:09.754989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc33f7-e2f9-4142-9345-c12bb7dfb1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:09.755626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc33f7-e2f9-4142-9345-c12bb7dfb1aa None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:09.755905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc33f7-e2f9-4142-9345-c12bb7dfb1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:09.756181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc33f7-e2f9-4142-9345-c12bb7dfb1aa None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:09.757117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc33f7-e2f9-4142-9345-c12bb7dfb1aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:09.759233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fc33f7-e2f9-4142-9345-c12bb7dfb1aa None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:09.766627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 385/778] 10.222.0.45 () {56 vars in 1087 bytes} [Thu Aug 4 02:05:09 2022] GET /identity/v3/services => generated 772 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:09.792364 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f7504-294e-4b73-80de-fdcd53bd4c15 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:09.831651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704ba02-53e0-40f4-99c4-a5bc3a30ee6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:09.832251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704ba02-53e0-40f4-99c4-a5bc3a30ee6a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:09.832539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704ba02-53e0-40f4-99c4-a5bc3a30ee6a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:09.832834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704ba02-53e0-40f4-99c4-a5bc3a30ee6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:09.833645 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704ba02-53e0-40f4-99c4-a5bc3a30ee6a None admin] RBAC: Adding query filter params (interface=public, service_id=50be4772503b476da8e5f1b9ebfb765e, region_id=RegionOne) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:09.833965 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704ba02-53e0-40f4-99c4-a5bc3a30ee6a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:09.835388 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1704ba02-53e0-40f4-99c4-a5bc3a30ee6a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:09.844011 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 394/779] 10.222.0.45 () {56 vars in 1251 bytes} [Thu Aug 4 02:05:09 2022] GET /identity/v3/endpoints?service_id=50be4772503b476da8e5f1b9ebfb765e&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:09.850190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc33f7-e2f9-4142-9345-c12bb7dfb1aa None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:09.940963 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:09.966585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc33f7-e2f9-4142-9345-c12bb7dfb1aa None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:10.003376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38ae79-7f73-46e5-97a8-219d7d44160b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:10.004008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38ae79-7f73-46e5-97a8-219d7d44160b None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:10.004286 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38ae79-7f73-46e5-97a8-219d7d44160b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:10.004556 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38ae79-7f73-46e5-97a8-219d7d44160b None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:10.005262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38ae79-7f73-46e5-97a8-219d7d44160b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:10.006676 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38ae79-7f73-46e5-97a8-219d7d44160b None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:10.013238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 386/780] 10.222.0.45 () {56 vars in 1087 bytes} [Thu Aug 4 02:05:09 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:10.030657 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20f6ce-5bd0-434e-a894-9f4b53b8ccf7 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:10.031181 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20f6ce-5bd0-434e-a894-9f4b53b8ccf7 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:10.031256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20f6ce-5bd0-434e-a894-9f4b53b8ccf7 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:10.032983 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 395/781] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:10.039296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf769da4-501b-47c3-a059-f5fdbb57526c None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:10.039647 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf769da4-501b-47c3-a059-f5fdbb57526c None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:10.039843 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf769da4-501b-47c3-a059-f5fdbb57526c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:10.060127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-cf769da4-501b-47c3-a059-f5fdbb57526c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:10.061452 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-cf769da4-501b-47c3-a059-f5fdbb57526c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:10.144574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 387/782] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:10.175655 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20f6ce-5bd0-434e-a894-9f4b53b8ccf7 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:10.278669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:10.289452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20f6ce-5bd0-434e-a894-9f4b53b8ccf7 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:10.326030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a1af7-cddd-41c5-a22d-a11d451e6341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:10.326726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a1af7-cddd-41c5-a22d-a11d451e6341 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:10.327004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a1af7-cddd-41c5-a22d-a11d451e6341 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:10.327319 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a1af7-cddd-41c5-a22d-a11d451e6341 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:10.338821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a1af7-cddd-41c5-a22d-a11d451e6341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:05:10.343851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-548a1af7-cddd-41c5-a22d-a11d451e6341 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:05:10.343851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:10.343851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:10.343851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:10.343851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:10.343851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:10.343851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:10.343851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:10.343851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:10.343851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:10.343851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:10.343851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:10.343851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:05:10.343851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:05:10.343851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:05:10.343851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:05:10.346063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:05:10.346063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:05:10.346063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:05:10.346063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:05:10.346063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:10.346063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:10.346063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:10.346063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:10.346063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:10.346063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:10.346063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:10.346063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:10.346063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:10.346063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:10.346063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:10.346063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:10.348699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:10.348699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:10.348699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:10.348699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:10.348699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:05:10.348699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:05:10.348699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:05:10.348699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:05:10.348699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:05:10.348699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:05:10.348699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:05:10.348699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:10.351378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 396/783] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:05:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:10.355544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf769da4-501b-47c3-a059-f5fdbb57526c None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:10.458691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:10.494436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf769da4-501b-47c3-a059-f5fdbb57526c None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:10.530327 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c8e18-8ccf-4b97-b110-82efda16904b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:10.530956 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c8e18-8ccf-4b97-b110-82efda16904b None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:10.531289 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c8e18-8ccf-4b97-b110-82efda16904b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:10.531570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c8e18-8ccf-4b97-b110-82efda16904b None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:10.532291 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c8e18-8ccf-4b97-b110-82efda16904b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:10.532593 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c8e18-8ccf-4b97-b110-82efda16904b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:10.534247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c8e18-8ccf-4b97-b110-82efda16904b None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:10.543579 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 388/784] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:05:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:10.549392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a1af7-cddd-41c5-a22d-a11d451e6341 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:10.672475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a1af7-cddd-41c5-a22d-a11d451e6341 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:10.724194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b916d3-addd-425b-a951-a85dbc8ee3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:10.724781 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b916d3-addd-425b-a951-a85dbc8ee3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:10.725060 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b916d3-addd-425b-a951-a85dbc8ee3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:10.725331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b916d3-addd-425b-a951-a85dbc8ee3e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:10.741771 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b916d3-addd-425b-a951-a85dbc8ee3e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:10.743217 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b916d3-addd-425b-a951-a85dbc8ee3e5 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:10.749576 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0921378-abb1-4374-9383-46a49aa31d2c None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:10.749860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0921378-abb1-4374-9383-46a49aa31d2c None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:10.750135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0921378-abb1-4374-9383-46a49aa31d2c None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:10.751716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 389/785] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:05:10.756648 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1f403-9f1f-4328-b4c8-2883f95fb330 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:10.756940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1f403-9f1f-4328-b4c8-2883f95fb330 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:10.757216 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1f403-9f1f-4328-b4c8-2883f95fb330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:10.759041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-37b916d3-addd-425b-a951-a85dbc8ee3e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:10.759041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:10.759041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:10.759041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:10.759041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:10.759041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:10.759041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:10.759041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:10.759041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:10.759041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:10.759041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:10.759041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:10.759041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:10.759041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:10.759041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:10.761216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:10.761216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:10.761216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:10.761216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:10.761216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:10.761216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:10.761216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:10.761216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:10.761216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:10.761216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:10.761216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:10.761216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:10.761216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:10.761216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:10.761216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:10.761216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:10.761216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:10.763433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:10.763433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:10.763433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:10.763433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:10.763433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 397/786] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:10.769229 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b916d3-addd-425b-a951-a85dbc8ee3e5 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:10.775340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:10.780994 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-b2c1f403-9f1f-4328-b4c8-2883f95fb330 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:10.782253 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-b2c1f403-9f1f-4328-b4c8-2883f95fb330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:10.851729 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 390/787] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 02:05:10.873531 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1f403-9f1f-4328-b4c8-2883f95fb330 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:10.892780 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b916d3-addd-425b-a951-a85dbc8ee3e5 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:10.944495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630f441-687e-4ecb-b8b5-41e4f0fa2e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:10.945277 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630f441-687e-4ecb-b8b5-41e4f0fa2e68 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:10.945556 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630f441-687e-4ecb-b8b5-41e4f0fa2e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:10.945825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9630f441-687e-4ecb-b8b5-41e4f0fa2e68 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:10.946784 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630f441-687e-4ecb-b8b5-41e4f0fa2e68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:10.947571 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630f441-687e-4ecb-b8b5-41e4f0fa2e68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:10.949225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9630f441-687e-4ecb-b8b5-41e4f0fa2e68 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:10.950840 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:10.972181 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 398/788] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:10.977909 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630f441-687e-4ecb-b8b5-41e4f0fa2e68 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:11.005416 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1f403-9f1f-4328-b4c8-2883f95fb330 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:11.040560 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b07b2b-e82f-42ee-a7b6-ac95345bd7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:11.041828 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b07b2b-e82f-42ee-a7b6-ac95345bd7ca None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:11.042265 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b07b2b-e82f-42ee-a7b6-ac95345bd7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:11.042663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b07b2b-e82f-42ee-a7b6-ac95345bd7ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:11.053578 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b07b2b-e82f-42ee-a7b6-ac95345bd7ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:05:11.058923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-29b07b2b-e82f-42ee-a7b6-ac95345bd7ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:05:11.058923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:11.058923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:11.058923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:11.058923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:11.058923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:11.058923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:11.058923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:11.058923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:11.058923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:11.058923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:11.058923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:11.058923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:05:11.058923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:05:11.058923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:05:11.058923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:05:11.060065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:05:11.060065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:05:11.060065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:05:11.060065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:05:11.060065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:11.060065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:11.060065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:11.060065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:11.060065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:11.060065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:11.060065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:11.060065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:11.060065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:11.060065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:11.060065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:11.060065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:11.061444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:11.061444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:11.061444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:11.061444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:11.061444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:05:11.061444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:05:11.061444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:05:11.061444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:05:11.061444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:05:11.061444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:05:11.061444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:05:11.061444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:11.062079 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 391/789] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:05:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:11.069167 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b07b2b-e82f-42ee-a7b6-ac95345bd7ca None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:11.183689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b07b2b-e82f-42ee-a7b6-ac95345bd7ca None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:11.206028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:11.215157 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:11.236506 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8daad-6d6b-467f-9f43-bbc490b24eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:11.238640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8daad-6d6b-467f-9f43-bbc490b24eab None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:11.239145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8daad-6d6b-467f-9f43-bbc490b24eab None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:11.239623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c8daad-6d6b-467f-9f43-bbc490b24eab None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:11.240547 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8daad-6d6b-467f-9f43-bbc490b24eab None admin] RBAC: Adding query filter params (name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:11.241476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8daad-6d6b-467f-9f43-bbc490b24eab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:11.243127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c8daad-6d6b-467f-9f43-bbc490b24eab None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:11.253180 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 392/790] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:05:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:11.259732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8daad-6d6b-467f-9f43-bbc490b24eab None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:11.292226 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9630f441-687e-4ecb-b8b5-41e4f0fa2e68 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:11.354167 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc610a9c-967d-493d-959e-9589b312ae37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:11.354812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc610a9c-967d-493d-959e-9589b312ae37 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:11.355093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc610a9c-967d-493d-959e-9589b312ae37 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:11.355364 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc610a9c-967d-493d-959e-9589b312ae37 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:11.366607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc610a9c-967d-493d-959e-9589b312ae37 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:11.368058 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc610a9c-967d-493d-959e-9589b312ae37 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:11.380644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-cc610a9c-967d-493d-959e-9589b312ae37 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 02:05:11.380644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:11.380644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:11.380644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:11.380644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:11.380644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:11.380644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:11.380644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:11.380644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:11.380644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:11.380644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:11.380644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:11.380644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:05:11.380644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:05:11.380644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:05:11.380644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:11.381758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:11.382730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:11.382730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:11.382730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:05:11.382730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:05:11.382730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:05:11.382730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:05:11.382730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:05:11.382730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:05:11.382730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 02:05:11.382730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:11.382730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 399/791] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:05:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:11.387607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc610a9c-967d-493d-959e-9589b312ae37 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:11.453981 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c8daad-6d6b-467f-9f43-bbc490b24eab None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:11.490718 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2610a0b-ceea-47c7-b0b0-51ea0fd5fdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:11.491286 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2610a0b-ceea-47c7-b0b0-51ea0fd5fdf6 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:11.491563 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2610a0b-ceea-47c7-b0b0-51ea0fd5fdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:11.491834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2610a0b-ceea-47c7-b0b0-51ea0fd5fdf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:11.506625 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2610a0b-ceea-47c7-b0b0-51ea0fd5fdf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:11.508102 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2610a0b-ceea-47c7-b0b0-51ea0fd5fdf6 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:11.526108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-d2610a0b-ceea-47c7-b0b0-51ea0fd5fdf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:11.526108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:11.526108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:11.526108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:11.526108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:11.526108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:11.526108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:11.526108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:11.526108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:11.526108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:11.526108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:11.526108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:11.526108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:11.526108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:11.526108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:11.527094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:11.527094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:11.527094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:11.527094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:11.527094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:11.527094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:11.527094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:11.527094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:11.527094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:11.527094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:11.527094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:11.527094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:11.527094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:11.527094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:11.527094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:11.527094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:11.527094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:11.528007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:11.528007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:11.528007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:11.528007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:11.528007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 393/792] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:11.533511 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2610a0b-ceea-47c7-b0b0-51ea0fd5fdf6 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:11.579281 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:11.586605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc610a9c-967d-493d-959e-9589b312ae37 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:11.638411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:11.663469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fec1b-7b6d-41b1-952a-055a0f0ae2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:11.664383 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fec1b-7b6d-41b1-952a-055a0f0ae2aa None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:11.664899 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fec1b-7b6d-41b1-952a-055a0f0ae2aa None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:11.665353 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fec1b-7b6d-41b1-952a-055a0f0ae2aa None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:11.666824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1fec1b-7b6d-41b1-952a-055a0f0ae2aa None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:11.666824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1fec1b-7b6d-41b1-952a-055a0f0ae2aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:11.668048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1fec1b-7b6d-41b1-952a-055a0f0ae2aa None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:11.672169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2610a0b-ceea-47c7-b0b0-51ea0fd5fdf6 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:11.703917 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 400/793] 10.222.0.45 () {56 vars in 1135 bytes} [Thu Aug 4 02:05:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:11.710684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fec1b-7b6d-41b1-952a-055a0f0ae2aa None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:11.743563 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bedcb-20e5-4d03-9f40-3e9b05bc6dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:11.744110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bedcb-20e5-4d03-9f40-3e9b05bc6dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:11.744392 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bedcb-20e5-4d03-9f40-3e9b05bc6dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:11.744684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3bedcb-20e5-4d03-9f40-3e9b05bc6dc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:11.745385 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bedcb-20e5-4d03-9f40-3e9b05bc6dc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:11.745689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bedcb-20e5-4d03-9f40-3e9b05bc6dc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:11.747064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3bedcb-20e5-4d03-9f40-3e9b05bc6dc5 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:11.771213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 394/794] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:11.777013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bedcb-20e5-4d03-9f40-3e9b05bc6dc5 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:11.894328 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fec1b-7b6d-41b1-952a-055a0f0ae2aa None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:11.938345 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145789a5-d02e-4cd6-8831-dd5902f1db99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:11.938995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145789a5-d02e-4cd6-8831-dd5902f1db99 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:11.939280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145789a5-d02e-4cd6-8831-dd5902f1db99 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:11.939551 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145789a5-d02e-4cd6-8831-dd5902f1db99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:11.955170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:11.956196 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145789a5-d02e-4cd6-8831-dd5902f1db99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:11.958170 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145789a5-d02e-4cd6-8831-dd5902f1db99 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:11.966305 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3bedcb-20e5-4d03-9f40-3e9b05bc6dc5 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:11.977753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-145789a5-d02e-4cd6-8831-dd5902f1db99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:11.977753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:11.977753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:11.977753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:11.977753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:11.977753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:11.977753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:11.977753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:11.977753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:11.977753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:11.977753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:11.977753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:11.977753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:11.977753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:11.977753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:11.978674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:11.978674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:11.978674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:11.978674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:11.978674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:11.978674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:11.978674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:11.978674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:11.978674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:11.978674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:11.978674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:11.978674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:11.978674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:11.978674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:11.978674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:11.978674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:11.978674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:11.980676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:11.980676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:11.980676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:11.980676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:11.980676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 401/795] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:11.980676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71a3f0-c560-4b3f-874d-6dd1f8d300e8 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:11.981022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71a3f0-c560-4b3f-874d-6dd1f8d300e8 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:11.981022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71a3f0-c560-4b3f-874d-6dd1f8d300e8 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:11.986168 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 402/796] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:11.986168 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71a3f0-c560-4b3f-874d-6dd1f8d300e8 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:12.014519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eaa1c3-9a80-45a8-ba54-f8da5f1ad900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:12.015136 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eaa1c3-9a80-45a8-ba54-f8da5f1ad900 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:12.015416 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eaa1c3-9a80-45a8-ba54-f8da5f1ad900 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:12.015686 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eaa1c3-9a80-45a8-ba54-f8da5f1ad900 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:12.023719 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eaa1c3-9a80-45a8-ba54-f8da5f1ad900 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:12.025264 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eaa1c3-9a80-45a8-ba54-f8da5f1ad900 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:12.026802 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:12.039635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-e9eaa1c3-9a80-45a8-ba54-f8da5f1ad900 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 02:05:12.039635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:12.039635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:12.039635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:12.039635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:12.039635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:12.039635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:12.039635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:12.039635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:12.039635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:12.039635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:12.039635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:12.039635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:05:12.039635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:05:12.039635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:05:12.039635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:12.040675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:12.042649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:12.042649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:12.042649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:05:12.042649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:05:12.042649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:05:12.042649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:05:12.042649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:05:12.042649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:05:12.042649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 02:05:12.042649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:12.042649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 395/797] 10.222.0.45 () {56 vars in 1117 bytes} [Thu Aug 4 02:05:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:12.042649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130437d8-235c-4d93-a875-8a74555a1d58 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:12.042649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130437d8-235c-4d93-a875-8a74555a1d58 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:12.043345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130437d8-235c-4d93-a875-8a74555a1d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:12.062718 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-130437d8-235c-4d93-a875-8a74555a1d58 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:12.064021 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-130437d8-235c-4d93-a875-8a74555a1d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:12.103548 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71a3f0-c560-4b3f-874d-6dd1f8d300e8 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:12.138368 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a748d0-c78e-44c4-8714-8f9d4c860fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:12.139770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 396/798] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:12.139770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130437d8-235c-4d93-a875-8a74555a1d58 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:12.148846 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a748d0-c78e-44c4-8714-8f9d4c860fff None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:12.149473 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a748d0-c78e-44c4-8714-8f9d4c860fff None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:12.149963 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a748d0-c78e-44c4-8714-8f9d4c860fff None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:12.150904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a748d0-c78e-44c4-8714-8f9d4c860fff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:12.151432 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a748d0-c78e-44c4-8714-8f9d4c860fff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:12.153071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a748d0-c78e-44c4-8714-8f9d4c860fff None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:12.175380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 403/799] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:12.176601 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a748d0-c78e-44c4-8714-8f9d4c860fff None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:12.236578 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:12.250088 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130437d8-235c-4d93-a875-8a74555a1d58 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:12.278219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:12.286191 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a748d0-c78e-44c4-8714-8f9d4c860fff None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:12.289705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80574240-038f-4f2a-a46a-1d138b04775f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:12.290181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80574240-038f-4f2a-a46a-1d138b04775f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:12.290462 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80574240-038f-4f2a-a46a-1d138b04775f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:12.290732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80574240-038f-4f2a-a46a-1d138b04775f None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:12.291531 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80574240-038f-4f2a-a46a-1d138b04775f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:12.291830 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80574240-038f-4f2a-a46a-1d138b04775f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:12.293288 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80574240-038f-4f2a-a46a-1d138b04775f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:12.313265 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 397/800] 10.222.0.45 () {56 vars in 1149 bytes} [Thu Aug 4 02:05:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:12.314212 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80574240-038f-4f2a-a46a-1d138b04775f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:12.325938 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6735e-5287-4cb1-ac89-5d43c3b85c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:12.326945 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6735e-5287-4cb1-ac89-5d43c3b85c8c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:12.327429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6735e-5287-4cb1-ac89-5d43c3b85c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:12.327900 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6735e-5287-4cb1-ac89-5d43c3b85c8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:12.335689 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6735e-5287-4cb1-ac89-5d43c3b85c8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:12.337535 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe6735e-5287-4cb1-ac89-5d43c3b85c8c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:12.350021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-dfe6735e-5287-4cb1-ac89-5d43c3b85c8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:12.350021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:12.350021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:12.350021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:12.350021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:12.350021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:12.350021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:12.350021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:12.350021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:12.350021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:12.350021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:12.350021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:12.350021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:05:12.350021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:05:12.350021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:05:12.350921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:05:12.350921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:12.350921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:12.350921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:12.350921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:12.350921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:12.350921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:12.350921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:12.350921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:12.350921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:12.350921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:12.350921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:12.350921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:12.350921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:12.350921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:12.350921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:12.350921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:12.351801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:05:12.351801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:05:12.351801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:05:12.351801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:05:12.351801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:05:12.351801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:05:12.351801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:12.351801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:12.355214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 404/801] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:12.356384 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6735e-5287-4cb1-ac89-5d43c3b85c8c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:12.420790 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80574240-038f-4f2a-a46a-1d138b04775f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:12.455460 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d487d-ee86-448b-85df-6ca8ee468775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:12.456131 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d487d-ee86-448b-85df-6ca8ee468775 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:12.456408 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d487d-ee86-448b-85df-6ca8ee468775 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:12.456698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d487d-ee86-448b-85df-6ca8ee468775 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:12.464039 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6735e-5287-4cb1-ac89-5d43c3b85c8c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:12.467362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d487d-ee86-448b-85df-6ca8ee468775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:05:12.472514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-4f5d487d-ee86-448b-85df-6ca8ee468775 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:05:12.472514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:12.472514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:12.472514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:12.472514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:12.472514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:12.472514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:12.472514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:12.472514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:12.472514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:12.472514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:12.472514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:12.472514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:05:12.472514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:05:12.472514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:05:12.472514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:05:12.474407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:05:12.474407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:05:12.474407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:05:12.474407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:05:12.474407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:12.474407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:12.474407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:12.474407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:12.474407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:12.474407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:12.474407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:12.474407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:12.474407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:12.474407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:12.474407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:12.474407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:12.475894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:12.475894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:12.475894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:12.475894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:12.475894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:05:12.475894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:05:12.475894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:05:12.475894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:05:12.475894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:05:12.475894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:05:12.475894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:05:12.475894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:12.475894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 398/802] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:05:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:12.475894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d487d-ee86-448b-85df-6ca8ee468775 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:12.486590 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:12.509774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664c849-8f74-443f-aa52-786600263048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:12.510459 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664c849-8f74-443f-aa52-786600263048 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:12.510683 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664c849-8f74-443f-aa52-786600263048 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:12.510951 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664c849-8f74-443f-aa52-786600263048 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:12.523975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5664c849-8f74-443f-aa52-786600263048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:12.525472 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5664c849-8f74-443f-aa52-786600263048 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:12.540310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-5664c849-8f74-443f-aa52-786600263048 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:12.540310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:12.540310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:12.540310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:12.540310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:12.540310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:12.540310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:12.540310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:12.540310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:12.540310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:12.540310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:12.540310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:12.540310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:12.540310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:12.540310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:12.541536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:12.541536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:12.541536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:12.541536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:12.541536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:12.541536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:12.541536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:12.541536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:12.541536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:12.541536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:12.541536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:12.541536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:12.541536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:12.541536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:12.541536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:12.541536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:12.541536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:12.542433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:12.542433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:12.542433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:12.542433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:12.542433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 405/803] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:12.542702 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef4015-bb9e-4dd4-aa5b-c3307a608706 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:12.544210 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef4015-bb9e-4dd4-aa5b-c3307a608706 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:12.544210 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef4015-bb9e-4dd4-aa5b-c3307a608706 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:12.544369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 406/804] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:12.549068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef4015-bb9e-4dd4-aa5b-c3307a608706 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:12.559941 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:12.594109 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d487d-ee86-448b-85df-6ca8ee468775 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:12.632236 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbacf3-e233-4977-b321-539dde479417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:12.633234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbacf3-e233-4977-b321-539dde479417 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:12.633776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbacf3-e233-4977-b321-539dde479417 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:12.634246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbacf3-e233-4977-b321-539dde479417 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:12.635557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbacf3-e233-4977-b321-539dde479417 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:12.636066 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbacf3-e233-4977-b321-539dde479417 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:12.638340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbacf3-e233-4977-b321-539dde479417 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:12.654501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 399/805] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:05:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:12.654501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbacf3-e233-4977-b321-539dde479417 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:12.670860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef4015-bb9e-4dd4-aa5b-c3307a608706 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:12.707695 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433f219-75ab-4221-a950-0df6e8627a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:12.708316 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433f219-75ab-4221-a950-0df6e8627a99 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:12.708605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433f219-75ab-4221-a950-0df6e8627a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:12.708945 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433f219-75ab-4221-a950-0df6e8627a99 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:12.709726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433f219-75ab-4221-a950-0df6e8627a99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:12.710033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433f219-75ab-4221-a950-0df6e8627a99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:12.711456 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433f219-75ab-4221-a950-0df6e8627a99 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:12.722565 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 407/806] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:05:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:12.723968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0858a55-66a4-44ca-9a4a-7450b05db3c9 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:12.724249 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0858a55-66a4-44ca-9a4a-7450b05db3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:12.724528 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0858a55-66a4-44ca-9a4a-7450b05db3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:12.731055 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:12.742757 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c0858a55-66a4-44ca-9a4a-7450b05db3c9 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:12.743966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c0858a55-66a4-44ca-9a4a-7450b05db3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:12.766372 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbacf3-e233-4977-b321-539dde479417 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:12.803441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d499c-a4ae-421c-90d2-2f0a537abdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:12.803972 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d499c-a4ae-421c-90d2-2f0a537abdb9 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:12.804255 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d499c-a4ae-421c-90d2-2f0a537abdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:12.804534 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d499c-a4ae-421c-90d2-2f0a537abdb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:12.805258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d499c-a4ae-421c-90d2-2f0a537abdb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:12.805563 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d499c-a4ae-421c-90d2-2f0a537abdb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:12.806933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d499c-a4ae-421c-90d2-2f0a537abdb9 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:12.815005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:12.819219 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 400/807] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:12.820015 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d499c-a4ae-421c-90d2-2f0a537abdb9 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:12.832812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 408/808] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:12.832812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a59ad-151c-4fb9-8673-1e428b42aef0 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:12.832812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a59ad-151c-4fb9-8673-1e428b42aef0 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:12.832812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a59ad-151c-4fb9-8673-1e428b42aef0 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:12.832812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 409/809] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:12.832812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a59ad-151c-4fb9-8673-1e428b42aef0 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:12.928083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d499c-a4ae-421c-90d2-2f0a537abdb9 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:12.932215 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a59ad-151c-4fb9-8673-1e428b42aef0 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:12.954367 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:12.963994 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b70c3-405a-43f0-aea6-580dcc820ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:12.964685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b70c3-405a-43f0-aea6-580dcc820ca9 None admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:12.964971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b70c3-405a-43f0-aea6-580dcc820ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:12.965248 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b70c3-405a-43f0-aea6-580dcc820ca9 None admin] PATH_INFO: `/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/155a7276efab426cb34175683c475f33/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:12.966875 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1a4cb-a4b0-443c-9468-c4e171610187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:12.967163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b70c3-405a-43f0-aea6-580dcc820ca9 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b8caff828a4f5289094fb2ae2278f7, user_id=155a7276efab426cb34175683c475f33, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:12.967504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1a4cb-a4b0-443c-9468-c4e171610187 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:12.967782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1a4cb-a4b0-443c-9468-c4e171610187 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:12.968052 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1a4cb-a4b0-443c-9468-c4e171610187 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:12.968592 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b70c3-405a-43f0-aea6-580dcc820ca9 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:12.980246 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1a4cb-a4b0-443c-9468-c4e171610187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:12.981998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b1a4cb-a4b0-443c-9468-c4e171610187 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:12.982332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 401/810] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:05:12 2022] PUT /identity/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/155a7276efab426cb34175683c475f33/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:05:12.983133 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b70c3-405a-43f0-aea6-580dcc820ca9 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:13.006646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-46b1a4cb-a4b0-443c-9468-c4e171610187 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:13.006646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:13.006646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:13.006646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:13.006646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:13.006646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:13.006646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:13.006646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:13.006646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:13.006646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:13.006646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:13.006646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:13.006646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:13.006646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:13.006646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:13.007894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:13.007894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:13.007894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:13.007894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:13.007894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:13.007894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:13.007894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:13.007894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:13.007894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:13.007894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:13.007894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:13.007894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:13.007894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:13.007894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:13.007894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:13.007894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:13.007894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:13.009141 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:13.009141 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:13.009141 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:13.009141 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:13.009141 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 410/811] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:13.009141 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0110a-2d61-4e61-83b0-ae22641e6498 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:13.009491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0110a-2d61-4e61-83b0-ae22641e6498 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:13.009770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0110a-2d61-4e61-83b0-ae22641e6498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:13.029288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-21e0110a-2d61-4e61-83b0-ae22641e6498 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:13.030565 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-21e0110a-2d61-4e61-83b0-ae22641e6498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:13.086329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:13.109708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b70c3-405a-43f0-aea6-580dcc820ca9 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:13.113224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 411/812] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:13.113520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0110a-2d61-4e61-83b0-ae22641e6498 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:13.171838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce69e5b-2efc-4850-adf5-f664c647b013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:13.173355 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce69e5b-2efc-4850-adf5-f664c647b013 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:13.173639 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce69e5b-2efc-4850-adf5-f664c647b013 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:13.173906 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce69e5b-2efc-4850-adf5-f664c647b013 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:13.181228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce69e5b-2efc-4850-adf5-f664c647b013 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:13.182687 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce69e5b-2efc-4850-adf5-f664c647b013 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:13.195258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-bce69e5b-2efc-4850-adf5-f664c647b013 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:13.195258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:13.195258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:13.195258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:13.195258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:13.195258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:13.195258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:13.195258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:13.195258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:13.195258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:13.195258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:13.195258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:13.195258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:05:13.195258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:05:13.195258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:05:13.196313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:05:13.196313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:13.196313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:13.196313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:13.196313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:13.196313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:13.196313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:13.196313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:13.196313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:13.196313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:13.196313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:13.196313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:13.196313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:13.196313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:13.196313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:13.196313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:13.196313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:13.197435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:05:13.197435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:05:13.197435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:05:13.197435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:05:13.197435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:05:13.197435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:05:13.197435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:13.197435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:13.199214 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 402/813] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:13.200129 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce69e5b-2efc-4850-adf5-f664c647b013 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:13.249350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:13.253830 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0110a-2d61-4e61-83b0-ae22641e6498 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:13.291870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e042be-09cd-424e-924c-7dd6deb5c630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:13.292547 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e042be-09cd-424e-924c-7dd6deb5c630 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:13.292813 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e042be-09cd-424e-924c-7dd6deb5c630 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:13.293093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e042be-09cd-424e-924c-7dd6deb5c630 None admin] PATH_INFO: `/v3/services/network` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:13.293738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e042be-09cd-424e-924c-7dd6deb5c630 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:13.295137 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e042be-09cd-424e-924c-7dd6deb5c630 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:13.305821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-f8e042be-09cd-424e-924c-7dd6deb5c630 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 02:05:13.305821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:13.305821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:13.305821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:13.305821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:13.305821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:13.305821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:13.305821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:13.305821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 02:05:13.305821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:13.305821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:13.305821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:13.305821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:13.305821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:13.305821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:13.305821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:13.305821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:13.306925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:13.306925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:13.306925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:13.306925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:13.306925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:13.306925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:13.306925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:13.306925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:13.306925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:13.306925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:13.306925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:13.306925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:13.306925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:13.306925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:13.306925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:13.306925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:13.307778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:13.307778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:13.307778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:13.307778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:13.307778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:13.307778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:13.307778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:13.307778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:13.307778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:13.307778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:13.307778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:13.307778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:13.307778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 04 02:05:13.307778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:13.307778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 412/814] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:13 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:13.308576 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e042be-09cd-424e-924c-7dd6deb5c630 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:13.321458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce69e5b-2efc-4850-adf5-f664c647b013 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:13.357113 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:13.359924 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ae261-ef62-4f68-a509-cae41895d201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:13.360811 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ae261-ef62-4f68-a509-cae41895d201 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:13.361168 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ae261-ef62-4f68-a509-cae41895d201 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:13.361510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ae261-ef62-4f68-a509-cae41895d201 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:13.362236 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ae261-ef62-4f68-a509-cae41895d201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:13.362546 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ae261-ef62-4f68-a509-cae41895d201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:13.363971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ae261-ef62-4f68-a509-cae41895d201 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:13.381333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 403/815] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:13.381333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ae261-ef62-4f68-a509-cae41895d201 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:13.418182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e042be-09cd-424e-924c-7dd6deb5c630 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:13.455500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3098e8-fd38-459f-9f0c-3297eac5da2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:13.456796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3098e8-fd38-459f-9f0c-3297eac5da2a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:13.457081 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3098e8-fd38-459f-9f0c-3297eac5da2a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:13.457352 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3098e8-fd38-459f-9f0c-3297eac5da2a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:13.457977 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3098e8-fd38-459f-9f0c-3297eac5da2a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:13.459347 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3098e8-fd38-459f-9f0c-3297eac5da2a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:13.468282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-4b3098e8-fd38-459f-9f0c-3297eac5da2a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 02:05:13.468282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:13.468282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:13.468282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:13.468282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:13.468282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:13.468282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:13.468282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:13.468282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 02:05:13.468282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:13.468282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:13.468282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:13.468282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:13.468282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:13.468282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:13.468282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:13.469242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:13.469242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:13.469242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:13.469242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:13.469242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:13.469242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:13.469242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:13.469242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:13.469242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:13.469242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:13.469242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:13.469242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:13.469242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:13.469242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:13.469242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:13.469242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:13.470100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:13.470100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:13.470100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:13.470100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:13.470100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:13.470100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:13.470100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:13.470100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:13.470100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:13.470100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:13.470100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:13.470100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:13.470100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:13.470100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 04 02:05:13.470100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:13.470100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 413/816] 10.222.0.45 () {56 vars in 1129 bytes} [Thu Aug 4 02:05:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:13.470977 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3098e8-fd38-459f-9f0c-3297eac5da2a None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:13.489432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ae261-ef62-4f68-a509-cae41895d201 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:13.497995 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:13.534094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82f600-d268-437a-86cb-126727ee701d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:13.534872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82f600-d268-437a-86cb-126727ee701d None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:13.535155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82f600-d268-437a-86cb-126727ee701d None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:13.535752 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82f600-d268-437a-86cb-126727ee701d None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:13.536984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82f600-d268-437a-86cb-126727ee701d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:13.537296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82f600-d268-437a-86cb-126727ee701d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:13.539513 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82f600-d268-437a-86cb-126727ee701d None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:13.558997 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 404/817] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:05:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:13.558997 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82f600-d268-437a-86cb-126727ee701d None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:13.579926 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3098e8-fd38-459f-9f0c-3297eac5da2a None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:13.590732 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:13.617891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21214b-91fb-4632-a998-7734a3258e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:13.618432 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21214b-91fb-4632-a998-7734a3258e4f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:13.618714 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21214b-91fb-4632-a998-7734a3258e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:13.618992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21214b-91fb-4632-a998-7734a3258e4f None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:13.619673 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21214b-91fb-4632-a998-7734a3258e4f None admin] RBAC: Adding query filter params (name=network) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:13.619978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21214b-91fb-4632-a998-7734a3258e4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:13.621384 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21214b-91fb-4632-a998-7734a3258e4f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:13.631360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 414/818] 10.222.0.45 () {56 vars in 1112 bytes} [Thu Aug 4 02:05:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:13.631360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21214b-91fb-4632-a998-7734a3258e4f None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:13.673363 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82f600-d268-437a-86cb-126727ee701d None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:13.712221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5029e50-68e1-44cf-a0c2-6a524b700e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:13.712914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5029e50-68e1-44cf-a0c2-6a524b700e5e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:13.713234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5029e50-68e1-44cf-a0c2-6a524b700e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:13.713567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5029e50-68e1-44cf-a0c2-6a524b700e5e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:13.722050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5029e50-68e1-44cf-a0c2-6a524b700e5e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:13.723523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5029e50-68e1-44cf-a0c2-6a524b700e5e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:13.735800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-d5029e50-68e1-44cf-a0c2-6a524b700e5e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:05:13.735800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:13.735800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:13.735800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:13.735800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:13.735800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:13.735800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:13.735800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:13.735800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:13.735800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:13.735800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:13.735800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:13.735800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:05:13.735800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:05:13.735800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:05:13.735800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:13.736907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:13.739362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:13.739362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:13.739362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:05:13.739362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:05:13.739362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:05:13.739362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:05:13.739362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:05:13.739362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:05:13.739362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:05:13.739362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:13.739362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 405/819] 10.222.0.45 () {56 vars in 1093 bytes} [Thu Aug 4 02:05:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:13.739362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5029e50-68e1-44cf-a0c2-6a524b700e5e None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:13.746136 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21214b-91fb-4632-a998-7734a3258e4f None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:13.768390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:13.782917 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec4b11-0f4c-41e8-a0ed-fef24c743fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:13.783535 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec4b11-0f4c-41e8-a0ed-fef24c743fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:13.783815 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec4b11-0f4c-41e8-a0ed-fef24c743fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:13.784120 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec4b11-0f4c-41e8-a0ed-fef24c743fb4 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:13.784836 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec4b11-0f4c-41e8-a0ed-fef24c743fb4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:13.785141 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec4b11-0f4c-41e8-a0ed-fef24c743fb4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:13.786497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec4b11-0f4c-41e8-a0ed-fef24c743fb4 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:13.795636 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 415/820] 10.222.0.45 () {56 vars in 1124 bytes} [Thu Aug 4 02:05:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:13.795636 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec4b11-0f4c-41e8-a0ed-fef24c743fb4 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:13.829927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:13.852232 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5029e50-68e1-44cf-a0c2-6a524b700e5e None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:13.885893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53627a4-46ac-4a51-8ed7-972024334524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:13.886562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53627a4-46ac-4a51-8ed7-972024334524 None admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:13.886839 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53627a4-46ac-4a51-8ed7-972024334524 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:13.887111 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53627a4-46ac-4a51-8ed7-972024334524 None admin] PATH_INFO: `/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/e54e713b52e84393b018807f8153922e/roles/977824a8c6a24c00af2f6dabd0e2a6f0` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:13.898261 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec4b11-0f4c-41e8-a0ed-fef24c743fb4 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:13.904325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53627a4-46ac-4a51-8ed7-972024334524 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b8caff828a4f5289094fb2ae2278f7, user_id=e54e713b52e84393b018807f8153922e, role_id=977824a8c6a24c00af2f6dabd0e2a6f0)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:13.905892 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53627a4-46ac-4a51-8ed7-972024334524 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:13.920747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 406/821] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:05:13 2022] PUT /identity/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/e54e713b52e84393b018807f8153922e/roles/977824a8c6a24c00af2f6dabd0e2a6f0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:05:13.921922 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53627a4-46ac-4a51-8ed7-972024334524 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:13.952110 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac1c9b-0ea6-4be5-968d-f94535e6f194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:13.952809 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac1c9b-0ea6-4be5-968d-f94535e6f194 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:13.953170 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac1c9b-0ea6-4be5-968d-f94535e6f194 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:13.953978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ac1c9b-0ea6-4be5-968d-f94535e6f194 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:13.954776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac1c9b-0ea6-4be5-968d-f94535e6f194 None admin] RBAC: Adding query filter params (type=network) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:13.955142 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac1c9b-0ea6-4be5-968d-f94535e6f194 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:13.957706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ac1c9b-0ea6-4be5-968d-f94535e6f194 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:13.968819 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 416/822] 10.222.0.45 () {56 vars in 1112 bytes} [Thu Aug 4 02:05:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:13.971326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac1c9b-0ea6-4be5-968d-f94535e6f194 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:14.020971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:14.053516 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53627a4-46ac-4a51-8ed7-972024334524 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:14.100682 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac3adc-4ffd-49f3-97e3-6972308be9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:14.101315 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac3adc-4ffd-49f3-97e3-6972308be9fd None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:14.101594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac3adc-4ffd-49f3-97e3-6972308be9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:14.101862 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac3adc-4ffd-49f3-97e3-6972308be9fd None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:14.102661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac3adc-4ffd-49f3-97e3-6972308be9fd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:14.102961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac3adc-4ffd-49f3-97e3-6972308be9fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:14.104399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac3adc-4ffd-49f3-97e3-6972308be9fd None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:14.116405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:14.119879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ac1c9b-0ea6-4be5-968d-f94535e6f194 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:14.131264 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 407/823] 10.222.0.45 () {56 vars in 1133 bytes} [Thu Aug 4 02:05:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:14.132354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac3adc-4ffd-49f3-97e3-6972308be9fd None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:14.168145 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12e289-15b1-4eb2-8eac-724c02aed733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:14.168716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12e289-15b1-4eb2-8eac-724c02aed733 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:14.168996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12e289-15b1-4eb2-8eac-724c02aed733 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:14.169269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f12e289-15b1-4eb2-8eac-724c02aed733 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:14.169950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12e289-15b1-4eb2-8eac-724c02aed733 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:14.170251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12e289-15b1-4eb2-8eac-724c02aed733 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:14.171619 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f12e289-15b1-4eb2-8eac-724c02aed733 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:14.177367 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 417/824] 10.222.0.45 () {56 vars in 1124 bytes} [Thu Aug 4 02:05:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:14.178252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12e289-15b1-4eb2-8eac-724c02aed733 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:14.249016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac3adc-4ffd-49f3-97e3-6972308be9fd None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:14.294379 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6836b6d-6093-46da-a62d-0f781f360455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:14.295006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6836b6d-6093-46da-a62d-0f781f360455 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:14.295287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6836b6d-6093-46da-a62d-0f781f360455 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:14.295614 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6836b6d-6093-46da-a62d-0f781f360455 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:14.296945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6836b6d-6093-46da-a62d-0f781f360455 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:14.298599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6836b6d-6093-46da-a62d-0f781f360455 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:14.306034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:14.308265 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f12e289-15b1-4eb2-8eac-724c02aed733 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:14.332709 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 408/825] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:05:14 2022] POST /identity/v3/endpoints => generated 337 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:05:14.333855 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6836b6d-6093-46da-a62d-0f781f360455 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:14.391685 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278d734-5ec2-43ac-8154-aefb3f4d3bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:14.392334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278d734-5ec2-43ac-8154-aefb3f4d3bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:14.392643 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278d734-5ec2-43ac-8154-aefb3f4d3bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:14.392906 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278d734-5ec2-43ac-8154-aefb3f4d3bb5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:14.463767 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:14.510979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6836b6d-6093-46da-a62d-0f781f360455 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:14.554939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278d734-5ec2-43ac-8154-aefb3f4d3bb5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:14.556858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278d734-5ec2-43ac-8154-aefb3f4d3bb5 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:14.563783 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee5721-4769-4b8a-a7ba-c0d8ae51131f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:14.564413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee5721-4769-4b8a-a7ba-c0d8ae51131f None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:14.564718 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee5721-4769-4b8a-a7ba-c0d8ae51131f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:14.564994 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee5721-4769-4b8a-a7ba-c0d8ae51131f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:14.566356 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee5721-4769-4b8a-a7ba-c0d8ae51131f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:14.567799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee5721-4769-4b8a-a7ba-c0d8ae51131f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:14.593134 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 409/826] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:05:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:05:14.703350 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 418/827] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:05:14 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:05:14.709822 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee5721-4769-4b8a-a7ba-c0d8ae51131f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:14.776395 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:14.846935 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee5721-4769-4b8a-a7ba-c0d8ae51131f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:14.886896 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffd92a-f043-4ae2-8fc9-3a76ce56ff02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:14.887930 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffd92a-f043-4ae2-8fc9-3a76ce56ff02 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:14.888413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffd92a-f043-4ae2-8fc9-3a76ce56ff02 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:14.888927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffd92a-f043-4ae2-8fc9-3a76ce56ff02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:14.902822 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbffd92a-f043-4ae2-8fc9-3a76ce56ff02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:14.904631 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbffd92a-f043-4ae2-8fc9-3a76ce56ff02 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:14.919835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-dbffd92a-f043-4ae2-8fc9-3a76ce56ff02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:14.919835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:14.919835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:14.919835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:14.919835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:14.919835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:14.919835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:14.919835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:14.919835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:14.919835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:14.919835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:14.919835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:14.919835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:14.919835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:14.919835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:14.920807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:14.920807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:14.920807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:14.920807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:14.920807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:14.920807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:14.920807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:14.920807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:14.920807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:14.920807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:14.920807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:14.920807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:14.920807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:14.920807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:14.920807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:14.920807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:14.920807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:14.921708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:14.921708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:14.921708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:14.921708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:14.923874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 410/828] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:14.929331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278d734-5ec2-43ac-8154-aefb3f4d3bb5 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:14.993479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:15.036727 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278d734-5ec2-43ac-8154-aefb3f4d3bb5 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:15.073237 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d5ed0-8e01-41ee-ab53-35c9c73d05f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:15.073855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d5ed0-8e01-41ee-ab53-35c9c73d05f9 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:15.074135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d5ed0-8e01-41ee-ab53-35c9c73d05f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:15.074405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d5ed0-8e01-41ee-ab53-35c9c73d05f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:15.075096 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d5ed0-8e01-41ee-ab53-35c9c73d05f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:15.075402 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d5ed0-8e01-41ee-ab53-35c9c73d05f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:15.076784 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d5ed0-8e01-41ee-ab53-35c9c73d05f9 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:15.089693 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 419/829] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:15.090179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ead2b-560c-42d1-b25f-07f88a781251 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:15.090472 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ead2b-560c-42d1-b25f-07f88a781251 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:15.090811 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ead2b-560c-42d1-b25f-07f88a781251 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:15.093376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 411/830] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:15.095080 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d5ed0-8e01-41ee-ab53-35c9c73d05f9 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:15.098048 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b2285-bc63-4f68-a271-e7f38cdde3cb None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:15.098328 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b2285-bc63-4f68-a271-e7f38cdde3cb None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:15.098601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4b2285-bc63-4f68-a271-e7f38cdde3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:15.117040 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-9e4b2285-bc63-4f68-a271-e7f38cdde3cb None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:15.118234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9e4b2285-bc63-4f68-a271-e7f38cdde3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:15.195140 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 412/831] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 02:05:15.218553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b2285-bc63-4f68-a271-e7f38cdde3cb None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:15.229961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d5ed0-8e01-41ee-ab53-35c9c73d05f9 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:15.245761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:15.259472 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:15.280868 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef32107-f092-499a-8b09-d4e17f3ff7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:15.281982 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef32107-f092-499a-8b09-d4e17f3ff7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:15.282706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef32107-f092-499a-8b09-d4e17f3ff7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:15.283163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef32107-f092-499a-8b09-d4e17f3ff7b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:15.294602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef32107-f092-499a-8b09-d4e17f3ff7b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:15.297272 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef32107-f092-499a-8b09-d4e17f3ff7b1 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:15.313927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-bef32107-f092-499a-8b09-d4e17f3ff7b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:15.313927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:15.313927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:15.313927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:15.313927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:15.313927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:15.313927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:15.313927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:15.313927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:15.313927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:15.313927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:15.313927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:15.313927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:05:15.313927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:05:15.313927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:05:15.315942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:05:15.315942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:15.315942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:15.315942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:15.315942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:15.315942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:15.315942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:15.315942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:15.315942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:15.315942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:15.315942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:15.315942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:15.315942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:15.315942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:15.315942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:15.315942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:15.315942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:15.316873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:05:15.316873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:05:15.316873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:05:15.316873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:05:15.316873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:05:15.316873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:05:15.316873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:15.316873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:15.318215 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 420/832] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:15.323659 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef32107-f092-499a-8b09-d4e17f3ff7b1 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:15.339012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4b2285-bc63-4f68-a271-e7f38cdde3cb None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:15.381474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7577b70-bd17-4ee7-9c8c-161ac2e81233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:15.382382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7577b70-bd17-4ee7-9c8c-161ac2e81233 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:15.382661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7577b70-bd17-4ee7-9c8c-161ac2e81233 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:15.383318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7577b70-bd17-4ee7-9c8c-161ac2e81233 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:15.398831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7577b70-bd17-4ee7-9c8c-161ac2e81233 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:05:15.404672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-a7577b70-bd17-4ee7-9c8c-161ac2e81233 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:05:15.404672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:15.404672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:15.404672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:15.404672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:15.404672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:15.404672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:15.404672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:15.404672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:15.404672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:15.404672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:15.404672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:15.404672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:05:15.404672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:05:15.404672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:05:15.404672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:05:15.405617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:05:15.405617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:05:15.405617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:05:15.405617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:05:15.405617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:15.405617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:15.405617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:15.405617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:15.405617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:15.405617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:15.405617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:15.405617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:15.405617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:15.405617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:15.405617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:15.405617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:15.406478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:15.406478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:15.406478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:15.406478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:15.406478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:05:15.406478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:05:15.406478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:05:15.406478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:05:15.406478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:05:15.406478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:05:15.406478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:05:15.406478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:15.406478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 413/833] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:05:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:15.411541 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7577b70-bd17-4ee7-9c8c-161ac2e81233 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:15.455916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef32107-f092-499a-8b09-d4e17f3ff7b1 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:15.500760 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:15.504936 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa088c0-43d1-4213-9d4c-346f928a9f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:15.505855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa088c0-43d1-4213-9d4c-346f928a9f63 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:15.507398 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa088c0-43d1-4213-9d4c-346f928a9f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:15.507886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa088c0-43d1-4213-9d4c-346f928a9f63 None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:15.509161 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa088c0-43d1-4213-9d4c-346f928a9f63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:15.509949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa088c0-43d1-4213-9d4c-346f928a9f63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:15.512319 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa088c0-43d1-4213-9d4c-346f928a9f63 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:15.520333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7577b70-bd17-4ee7-9c8c-161ac2e81233 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:15.532721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 421/834] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:05:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:15.534509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa088c0-43d1-4213-9d4c-346f928a9f63 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:15.558051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f1ad1-0f83-4434-968b-d81c72a95295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:15.558614 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f1ad1-0f83-4434-968b-d81c72a95295 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:15.558900 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f1ad1-0f83-4434-968b-d81c72a95295 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:15.559175 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f1ad1-0f83-4434-968b-d81c72a95295 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:15.559892 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f1ad1-0f83-4434-968b-d81c72a95295 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:15.560258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f1ad1-0f83-4434-968b-d81c72a95295 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:15.561726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962f1ad1-0f83-4434-968b-d81c72a95295 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:15.575941 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 414/835] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:05:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:15.577445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f1ad1-0f83-4434-968b-d81c72a95295 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:15.582176 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:15.644134 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa088c0-43d1-4213-9d4c-346f928a9f63 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:15.684412 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660efcc-4696-4d65-a66e-b4452e3a1691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:15.685443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660efcc-4696-4d65-a66e-b4452e3a1691 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:15.685949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660efcc-4696-4d65-a66e-b4452e3a1691 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:15.686436 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1660efcc-4696-4d65-a66e-b4452e3a1691 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:15.687434 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1660efcc-4696-4d65-a66e-b4452e3a1691 None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=c571406d2a3f40dab035a918399d6f6a) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:15.688302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1660efcc-4696-4d65-a66e-b4452e3a1691 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:15.689969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1660efcc-4696-4d65-a66e-b4452e3a1691 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:15.696676 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f1ad1-0f83-4434-968b-d81c72a95295 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:15.697360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 422/836] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:05:15 2022] GET /identity/v3/role_assignments?user.id=c571406d2a3f40dab035a918399d6f6a&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=35205394c8af4c52a61b12f92ba16644 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:15.742570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d782b-4294-48e7-a4fc-89d35c751c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:15.743607 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d782b-4294-48e7-a4fc-89d35c751c96 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:15.744091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d782b-4294-48e7-a4fc-89d35c751c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:15.744653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d782b-4294-48e7-a4fc-89d35c751c96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:15.757877 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d782b-4294-48e7-a4fc-89d35c751c96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:15.759833 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d782b-4294-48e7-a4fc-89d35c751c96 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:15.769203 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:15.778661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-9e6d782b-4294-48e7-a4fc-89d35c751c96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:15.778661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:15.778661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:15.778661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:15.778661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:15.778661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:15.778661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:15.778661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:15.778661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:15.778661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:15.778661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:15.778661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:15.778661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:15.778661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:15.778661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:15.779655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:15.779655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:15.779655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:15.779655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:15.779655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:15.779655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:15.779655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:15.779655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:15.779655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:15.779655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:15.779655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:15.779655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:15.779655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:15.779655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:15.779655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:15.779655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:15.779655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:15.780763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:15.780763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:15.780763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:15.780763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:15.780763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 415/837] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:15.787955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660efcc-4696-4d65-a66e-b4452e3a1691 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:15.912227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1660efcc-4696-4d65-a66e-b4452e3a1691 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:15.914937 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:15.950988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3706210f-39b1-42ec-b540-c34c35fec49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:15.951663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706210f-39b1-42ec-b540-c34c35fec49c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:15.951945 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706210f-39b1-42ec-b540-c34c35fec49c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:15.952219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3706210f-39b1-42ec-b540-c34c35fec49c None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:15.952950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3706210f-39b1-42ec-b540-c34c35fec49c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:15.953253 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3706210f-39b1-42ec-b540-c34c35fec49c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:15.954617 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3706210f-39b1-42ec-b540-c34c35fec49c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:15.968398 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 423/838] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:15.974357 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d782b-4294-48e7-a4fc-89d35c751c96 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:16.083710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d782b-4294-48e7-a4fc-89d35c751c96 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:16.119742 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14280e4f-5fbc-471f-855c-23e7aa747f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:16.120859 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14280e4f-5fbc-471f-855c-23e7aa747f29 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:16.121353 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14280e4f-5fbc-471f-855c-23e7aa747f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:16.121821 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14280e4f-5fbc-471f-855c-23e7aa747f29 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:16.130869 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14280e4f-5fbc-471f-855c-23e7aa747f29 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:16.132686 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14280e4f-5fbc-471f-855c-23e7aa747f29 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:16.145335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca3bc7-9f30-4ac0-84d0-284135916071 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:16.145639 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca3bc7-9f30-4ac0-84d0-284135916071 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:16.145898 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca3bc7-9f30-4ac0-84d0-284135916071 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:16.147411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-14280e4f-5fbc-471f-855c-23e7aa747f29 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 04 02:05:16.147411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:16.147411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:16.147411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:16.147411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:16.147411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:16.147411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:16.147411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:16.147411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:16.147411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:16.147411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:16.147411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:16.147411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:05:16.147411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:05:16.147411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:05:16.147411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:16.148755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:16.150107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:16.150107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:16.150107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:05:16.150107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:05:16.150107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:05:16.150107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:05:16.150107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:05:16.150107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:05:16.150107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 04 02:05:16.150107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:16.151733 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 424/839] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:16.152684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0d9bd-c704-4ab7-9a9d-a6a27ef9bf08 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:16.152976 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0d9bd-c704-4ab7-9a9d-a6a27ef9bf08 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:16.153248 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b0d9bd-c704-4ab7-9a9d-a6a27ef9bf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:16.154105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 416/840] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:05:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:16.161517 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14280e4f-5fbc-471f-855c-23e7aa747f29 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:16.176802 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-b6b0d9bd-c704-4ab7-9a9d-a6a27ef9bf08 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:16.179056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b6b0d9bd-c704-4ab7-9a9d-a6a27ef9bf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:16.232824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:16.268272 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 425/841] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 02:05:16.294794 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0d9bd-c704-4ab7-9a9d-a6a27ef9bf08 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:16.304595 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14280e4f-5fbc-471f-855c-23e7aa747f29 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:16.353651 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1929d26-3751-4ae5-ab17-16126d52cc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:16.355132 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1929d26-3751-4ae5-ab17-16126d52cc02 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:16.355816 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1929d26-3751-4ae5-ab17-16126d52cc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:16.356285 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1929d26-3751-4ae5-ab17-16126d52cc02 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:16.357599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1929d26-3751-4ae5-ab17-16126d52cc02 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:16.358622 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1929d26-3751-4ae5-ab17-16126d52cc02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:16.360812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1929d26-3751-4ae5-ab17-16126d52cc02 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:16.388047 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 417/842] 10.222.0.45 () {56 vars in 1135 bytes} [Thu Aug 4 02:05:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:16.393085 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1929d26-3751-4ae5-ab17-16126d52cc02 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:16.420361 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:16.429850 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b0d9bd-c704-4ab7-9a9d-a6a27ef9bf08 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:16.476545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58959f-f738-479b-b14d-2e47af70cd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:16.477339 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58959f-f738-479b-b14d-2e47af70cd72 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:16.477627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58959f-f738-479b-b14d-2e47af70cd72 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:16.478309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58959f-f738-479b-b14d-2e47af70cd72 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:16.493580 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58959f-f738-479b-b14d-2e47af70cd72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:05:16.499271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-1e58959f-f738-479b-b14d-2e47af70cd72 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:05:16.499271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:16.499271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:16.499271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:16.499271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:16.499271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:16.499271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:16.499271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:16.499271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:16.499271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:16.499271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:16.499271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:16.499271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:05:16.499271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:05:16.499271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:05:16.499271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:05:16.500250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:05:16.500250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:05:16.500250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:05:16.500250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:05:16.500250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:16.500250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:16.500250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:16.500250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:16.500250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:16.500250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:16.500250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:16.500250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:16.500250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:16.500250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:16.500250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:16.500250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:16.501116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:16.501116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:16.501116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:16.501116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:16.501116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:05:16.501116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:05:16.501116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:05:16.501116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:05:16.501116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:05:16.501116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:05:16.501116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:05:16.501116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:16.504707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 426/843] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:05:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:16.507277 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58959f-f738-479b-b14d-2e47af70cd72 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:16.536514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1929d26-3751-4ae5-ab17-16126d52cc02 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:16.561328 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:16.594038 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22404182-77b1-4535-af6a-65b7d20c9e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:16.595219 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22404182-77b1-4535-af6a-65b7d20c9e45 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:16.596930 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22404182-77b1-4535-af6a-65b7d20c9e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:16.597722 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22404182-77b1-4535-af6a-65b7d20c9e45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:16.672275 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58959f-f738-479b-b14d-2e47af70cd72 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:16.695678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22404182-77b1-4535-af6a-65b7d20c9e45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:16.697471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22404182-77b1-4535-af6a-65b7d20c9e45 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:16.727302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bd427-1b3e-4ac5-9396-d7fccf143416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:16.728281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bd427-1b3e-4ac5-9396-d7fccf143416 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:16.728802 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bd427-1b3e-4ac5-9396-d7fccf143416 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:16.729813 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bd427-1b3e-4ac5-9396-d7fccf143416 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:16.730990 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bd427-1b3e-4ac5-9396-d7fccf143416 None admin] RBAC: Adding query filter params (name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:16.731510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bd427-1b3e-4ac5-9396-d7fccf143416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:16.734017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bd427-1b3e-4ac5-9396-d7fccf143416 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:16.753462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 427/844] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:05:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:16.755545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bd427-1b3e-4ac5-9396-d7fccf143416 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:16.783426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:16.839750 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 418/845] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:05:16 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:05:16.845757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22404182-77b1-4535-af6a-65b7d20c9e45 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:16.865706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:16.885542 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bd427-1b3e-4ac5-9396-d7fccf143416 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:16.928009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dca32-3d27-4efa-a6ad-ee41db8e640f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:16.928683 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dca32-3d27-4efa-a6ad-ee41db8e640f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:16.929042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dca32-3d27-4efa-a6ad-ee41db8e640f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:16.929245 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dca32-3d27-4efa-a6ad-ee41db8e640f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:16.943031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dca32-3d27-4efa-a6ad-ee41db8e640f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:16.945154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dca32-3d27-4efa-a6ad-ee41db8e640f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:16.964774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-948dca32-3d27-4efa-a6ad-ee41db8e640f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:16.964774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:16.964774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:16.964774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:16.964774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:16.964774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:16.964774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:16.964774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:16.964774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:16.964774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:16.964774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:16.964774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:16.964774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:16.964774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:16.964774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:16.965708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:16.965708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:16.965708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:16.965708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:16.965708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:16.965708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:16.965708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:16.965708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:16.965708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:16.965708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:16.965708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:16.965708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:16.965708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:16.965708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:16.965708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:16.965708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:16.965708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:16.966621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:16.966621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:16.966621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:16.966621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:16.966621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 428/846] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:16.967769 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ac16f-95e6-4d96-be7c-aea6b8d528ea None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:16.967769 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ac16f-95e6-4d96-be7c-aea6b8d528ea None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:16.968022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530ac16f-95e6-4d96-be7c-aea6b8d528ea None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:16.974304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 429/847] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:16.975275 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ac16f-95e6-4d96-be7c-aea6b8d528ea None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:16.976207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22404182-77b1-4535-af6a-65b7d20c9e45 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:17.015097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6bfa3-113e-4ed3-94ed-486828472381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:17.015739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6bfa3-113e-4ed3-94ed-486828472381 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:17.016018 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6bfa3-113e-4ed3-94ed-486828472381 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:17.016288 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6bfa3-113e-4ed3-94ed-486828472381 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:17.027487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6bfa3-113e-4ed3-94ed-486828472381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:17.029003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e6bfa3-113e-4ed3-94ed-486828472381 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:17.046634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-e4e6bfa3-113e-4ed3-94ed-486828472381 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:17.046634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:17.046634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:17.046634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:17.046634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:17.046634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:17.046634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:17.046634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:17.046634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:17.046634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:17.046634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:17.046634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:17.046634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:17.046634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:17.046634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:17.047623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:17.047623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:17.047623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:17.047623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:17.047623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:17.047623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:17.047623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:17.047623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:17.047623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:17.047623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:17.047623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:17.047623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:17.047623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:17.047623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:17.047623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:17.047623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:17.047623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:17.048536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:17.048536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:17.048536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:17.048536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:17.048536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 419/848] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:17.049291 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68fbfa-5092-4337-9752-9c8f813d7a0c None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:17.049402 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68fbfa-5092-4337-9752-9c8f813d7a0c None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:17.049936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68fbfa-5092-4337-9752-9c8f813d7a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:17.058888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:17.068286 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-7e68fbfa-5092-4337-9752-9c8f813d7a0c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:17.069502 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7e68fbfa-5092-4337-9752-9c8f813d7a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:17.097836 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530ac16f-95e6-4d96-be7c-aea6b8d528ea None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:17.134145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:17.136661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a4430-66d4-4322-a7f3-c5da902e7862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:17.137773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a4430-66d4-4322-a7f3-c5da902e7862 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:17.138055 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a4430-66d4-4322-a7f3-c5da902e7862 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:17.138360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a4430-66d4-4322-a7f3-c5da902e7862 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:17.139131 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a4430-66d4-4322-a7f3-c5da902e7862 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:17.139436 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a4430-66d4-4322-a7f3-c5da902e7862 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:17.140057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 420/849] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:17.141226 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68fbfa-5092-4337-9752-9c8f813d7a0c None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:17.146930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a4430-66d4-4322-a7f3-c5da902e7862 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:17.163044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 430/850] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:17.164715 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a4430-66d4-4322-a7f3-c5da902e7862 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:17.247395 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68fbfa-5092-4337-9752-9c8f813d7a0c None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:17.270966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a4430-66d4-4322-a7f3-c5da902e7862 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:17.287739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0bd80-7b44-4ee8-9e64-aae7f39ce156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:17.288359 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0bd80-7b44-4ee8-9e64-aae7f39ce156 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:17.288664 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0bd80-7b44-4ee8-9e64-aae7f39ce156 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:17.288941 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0bd80-7b44-4ee8-9e64-aae7f39ce156 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:17.289712 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0bd80-7b44-4ee8-9e64-aae7f39ce156 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:17.290014 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0bd80-7b44-4ee8-9e64-aae7f39ce156 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:17.291970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0bd80-7b44-4ee8-9e64-aae7f39ce156 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:17.308498 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 421/851] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:17.308498 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0bd80-7b44-4ee8-9e64-aae7f39ce156 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:17.317717 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:17.326821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af227cdb-e05a-4688-8111-ab5b1034f7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:17.327459 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af227cdb-e05a-4688-8111-ab5b1034f7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:17.327739 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af227cdb-e05a-4688-8111-ab5b1034f7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:17.328010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af227cdb-e05a-4688-8111-ab5b1034f7a4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:17.328655 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af227cdb-e05a-4688-8111-ab5b1034f7a4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:17.330031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af227cdb-e05a-4688-8111-ab5b1034f7a4 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:17.340102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-af227cdb-e05a-4688-8111-ab5b1034f7a4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 02:05:17.340102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:17.340102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:17.340102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:17.340102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:17.340102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:17.340102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:17.340102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:17.340102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 02:05:17.340102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:17.340102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:17.340102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:17.340102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:17.340102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:17.340102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:17.340102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:17.340102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:17.341372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:17.341372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:17.341372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:17.341372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:17.341372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:17.341372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:17.341372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:17.341372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:17.341372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:17.341372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:17.341372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:17.341372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:17.341372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:17.341372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:17.341372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:17.341372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:17.342213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:17.342213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:17.342213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:17.342213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:17.342213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:17.342213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:17.342213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:17.342213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:17.342213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:17.342213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:17.342213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:17.342213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:17.342213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 02:05:17.342213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:17.342213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 431/852] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:05:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:17.342991 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af227cdb-e05a-4688-8111-ab5b1034f7a4 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:17.373644 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:17.418606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0bd80-7b44-4ee8-9e64-aae7f39ce156 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:17.453746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ebf6c6-73b1-4ba7-a364-7f2fcf9d4070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:17.454395 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebf6c6-73b1-4ba7-a364-7f2fcf9d4070 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:17.454663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebf6c6-73b1-4ba7-a364-7f2fcf9d4070 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:17.454937 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebf6c6-73b1-4ba7-a364-7f2fcf9d4070 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:17.459819 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af227cdb-e05a-4688-8111-ab5b1034f7a4 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:17.463041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ebf6c6-73b1-4ba7-a364-7f2fcf9d4070 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:17.464492 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ebf6c6-73b1-4ba7-a364-7f2fcf9d4070 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:17.476283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-e9ebf6c6-73b1-4ba7-a364-7f2fcf9d4070 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 02:05:17.476283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:17.476283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:17.476283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:17.476283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:17.476283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:17.476283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:17.476283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:17.476283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:17.476283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:17.476283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:17.476283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:17.476283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:05:17.476283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:05:17.476283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:05:17.476283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:17.477433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:17.478544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:17.478544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:17.478544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:05:17.478544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:05:17.478544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:05:17.478544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:05:17.478544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:05:17.478544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:05:17.478544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 02:05:17.478544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:17.482220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 422/853] 10.222.0.45 () {56 vars in 1117 bytes} [Thu Aug 4 02:05:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:17.483042 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ebf6c6-73b1-4ba7-a364-7f2fcf9d4070 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:17.501370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9033042-f7a4-497e-b1c9-b238759880bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:17.502225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9033042-f7a4-497e-b1c9-b238759880bb None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:17.503309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9033042-f7a4-497e-b1c9-b238759880bb None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:17.503587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9033042-f7a4-497e-b1c9-b238759880bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:17.510935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9033042-f7a4-497e-b1c9-b238759880bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:17.512356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9033042-f7a4-497e-b1c9-b238759880bb None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:17.524106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-d9033042-f7a4-497e-b1c9-b238759880bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:17.524106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:17.524106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:17.524106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:17.524106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:17.524106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:17.524106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:17.524106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:17.524106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:17.524106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:17.524106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:17.524106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:17.524106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:05:17.524106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:05:17.524106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:05:17.525069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:05:17.525069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:17.525069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:17.525069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:17.525069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:17.525069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:17.525069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:17.525069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:17.525069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:17.525069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:17.525069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:17.525069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:17.525069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:17.525069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:17.525069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:17.525069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:17.525069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:17.526549 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:05:17.526549 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:05:17.526549 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:05:17.526549 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:05:17.526549 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:05:17.526549 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:05:17.526549 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:17.526549 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:17.526549 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 432/854] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:17.526549 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9033042-f7a4-497e-b1c9-b238759880bb None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:17.582313 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:17.599967 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ebf6c6-73b1-4ba7-a364-7f2fcf9d4070 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:17.618667 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:17.636742 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9033042-f7a4-497e-b1c9-b238759880bb None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:17.637674 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d850cf-df3c-4c61-817f-6b1b1f08dea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:17.638358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d850cf-df3c-4c61-817f-6b1b1f08dea0 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:17.638705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d850cf-df3c-4c61-817f-6b1b1f08dea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:17.639044 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d850cf-df3c-4c61-817f-6b1b1f08dea0 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:17.639788 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d850cf-df3c-4c61-817f-6b1b1f08dea0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:17.640097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d850cf-df3c-4c61-817f-6b1b1f08dea0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:17.641511 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d850cf-df3c-4c61-817f-6b1b1f08dea0 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:17.647028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 423/855] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:05:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:17.647670 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d850cf-df3c-4c61-817f-6b1b1f08dea0 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:17.689294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36737ce-df9a-48f4-8eb5-6f01e173fcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:17.690280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36737ce-df9a-48f4-8eb5-6f01e173fcb4 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:17.690738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36737ce-df9a-48f4-8eb5-6f01e173fcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:17.691187 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36737ce-df9a-48f4-8eb5-6f01e173fcb4 None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:17.692173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36737ce-df9a-48f4-8eb5-6f01e173fcb4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:17.692776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36737ce-df9a-48f4-8eb5-6f01e173fcb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:17.694535 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36737ce-df9a-48f4-8eb5-6f01e173fcb4 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:17.715119 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 433/856] 10.222.0.45 () {56 vars in 1149 bytes} [Thu Aug 4 02:05:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:17.716395 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36737ce-df9a-48f4-8eb5-6f01e173fcb4 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:17.760384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d850cf-df3c-4c61-817f-6b1b1f08dea0 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:17.795306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470542f-671a-43af-8c26-82398dbf4799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:17.795923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470542f-671a-43af-8c26-82398dbf4799 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:17.796202 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470542f-671a-43af-8c26-82398dbf4799 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:17.796475 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f470542f-671a-43af-8c26-82398dbf4799 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:17.797252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f470542f-671a-43af-8c26-82398dbf4799 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:17.797557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f470542f-671a-43af-8c26-82398dbf4799 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:17.798981 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f470542f-671a-43af-8c26-82398dbf4799 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:17.812132 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 424/857] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:05:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:17.813285 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470542f-671a-43af-8c26-82398dbf4799 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:17.827188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36737ce-df9a-48f4-8eb5-6f01e173fcb4 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:17.840860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:17.859473 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:17.868394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcedf6-b8eb-4c8d-8760-59d9a992d4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:17.869080 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcedf6-b8eb-4c8d-8760-59d9a992d4d8 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:17.869362 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcedf6-b8eb-4c8d-8760-59d9a992d4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:17.869635 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcedf6-b8eb-4c8d-8760-59d9a992d4d8 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:17.870759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcedf6-b8eb-4c8d-8760-59d9a992d4d8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:17.871196 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcedf6-b8eb-4c8d-8760-59d9a992d4d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:17.872887 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcedf6-b8eb-4c8d-8760-59d9a992d4d8 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:17.879025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 434/858] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:05:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:17.879812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcedf6-b8eb-4c8d-8760-59d9a992d4d8 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:17.938431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f470542f-671a-43af-8c26-82398dbf4799 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:17.978510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792eca68-7393-4822-b67b-e0d4ad7424be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:17.979140 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792eca68-7393-4822-b67b-e0d4ad7424be None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:17.979684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792eca68-7393-4822-b67b-e0d4ad7424be None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:17.979684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792eca68-7393-4822-b67b-e0d4ad7424be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:18.001453 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcedf6-b8eb-4c8d-8760-59d9a992d4d8 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:18.042785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77393ac8-c0b4-4b7d-bb7f-e0e113b16307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:18.043344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77393ac8-c0b4-4b7d-bb7f-e0e113b16307 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:18.043939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77393ac8-c0b4-4b7d-bb7f-e0e113b16307 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:18.044227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77393ac8-c0b4-4b7d-bb7f-e0e113b16307 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:18.045544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77393ac8-c0b4-4b7d-bb7f-e0e113b16307 None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=155a7276efab426cb34175683c475f33) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:18.045654 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77393ac8-c0b4-4b7d-bb7f-e0e113b16307 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:18.047315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77393ac8-c0b4-4b7d-bb7f-e0e113b16307 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:18.054603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 435/859] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:05:17 2022] GET /identity/v3/role_assignments?user.id=155a7276efab426cb34175683c475f33&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=35205394c8af4c52a61b12f92ba16644 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:18.056536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49deb8-338d-4b62-a980-a370e94ef6bc None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:18.056796 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792eca68-7393-4822-b67b-e0d4ad7424be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:18.057066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49deb8-338d-4b62-a980-a370e94ef6bc None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:18.057340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a49deb8-338d-4b62-a980-a370e94ef6bc None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:18.058237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792eca68-7393-4822-b67b-e0d4ad7424be None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:18.058790 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 436/860] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:18.064882 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a339b56-ebb2-41e4-a68b-7d2fa860761c None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:18.065755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a339b56-ebb2-41e4-a68b-7d2fa860761c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:18.066028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a339b56-ebb2-41e4-a68b-7d2fa860761c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:18.086408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-2a339b56-ebb2-41e4-a68b-7d2fa860761c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:18.087633 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-2a339b56-ebb2-41e4-a68b-7d2fa860761c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:18.118688 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:18.125764 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:18.171357 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 437/861] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:18 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 02:05:18.190929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 425/862] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:05:17 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:05:18.193811 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a339b56-ebb2-41e4-a68b-7d2fa860761c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:18.196756 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792eca68-7393-4822-b67b-e0d4ad7424be None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:18.313830 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792eca68-7393-4822-b67b-e0d4ad7424be None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:18.324031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a339b56-ebb2-41e4-a68b-7d2fa860761c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:18.352963 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23785d7c-16ab-489f-9391-8c29c67e4b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:18.353668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23785d7c-16ab-489f-9391-8c29c67e4b8a None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:18.353954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23785d7c-16ab-489f-9391-8c29c67e4b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:18.354227 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23785d7c-16ab-489f-9391-8c29c67e4b8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:18.365919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23785d7c-16ab-489f-9391-8c29c67e4b8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:18.367470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23785d7c-16ab-489f-9391-8c29c67e4b8a None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:18.369123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142fb34-1508-406a-a2e6-9c84b5345936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:18.369761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:18.371513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142fb34-1508-406a-a2e6-9c84b5345936 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:18.371964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142fb34-1508-406a-a2e6-9c84b5345936 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:18.372376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142fb34-1508-406a-a2e6-9c84b5345936 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:18.373210 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142fb34-1508-406a-a2e6-9c84b5345936 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:18.374739 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142fb34-1508-406a-a2e6-9c84b5345936 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:18.386317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-23785d7c-16ab-489f-9391-8c29c67e4b8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:18.386317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:18.386317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:18.386317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:18.386317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:18.386317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:18.386317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:18.386317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:18.386317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:18.386317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:18.386317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:18.386317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:18.386317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:18.386317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:18.386317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:18.387436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:18.387436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:18.387436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:18.387436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:18.387436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:18.387436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:18.387436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:18.387436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:18.387436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:18.387436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:18.387436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:18.387436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:18.387436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:18.387436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:18.387436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:18.387436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:18.387436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:18.388497 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:18.388497 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:18.388497 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:18.388497 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:18.389540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-2142fb34-1508-406a-a2e6-9c84b5345936 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 02:05:18.389540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:18.389540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:18.389540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:18.389540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:18.389540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:18.389540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:18.389540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:18.389540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 02:05:18.389540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:18.389540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:18.389540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:18.389540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:18.389540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:18.389540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:18.389540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:18.390593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:18.390593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:18.390593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:18.390593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:18.390593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:18.390593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:18.390593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:18.390593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:18.390593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:18.390593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:18.390593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:18.390593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:18.390593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:18.390593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:18.390593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:18.390593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:18.391470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:18.391470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:18.391470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:18.391470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:18.391470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:18.391470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:18.391470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:18.391470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:18.391470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:18.391470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:18.391470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:18.391470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:18.391470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:18.391470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 02:05:18.391470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:18.392240 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 426/863] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:18.393110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23785d7c-16ab-489f-9391-8c29c67e4b8a None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:18.394128 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 438/864] 10.222.0.45 () {56 vars in 1132 bytes} [Thu Aug 4 02:05:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:18.400912 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142fb34-1508-406a-a2e6-9c84b5345936 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:18.435301 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:18.509183 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23785d7c-16ab-489f-9391-8c29c67e4b8a None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:18.517900 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142fb34-1508-406a-a2e6-9c84b5345936 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:18.546038 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e8af5-b60c-4631-a227-a85ce60604fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:18.547004 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e8af5-b60c-4631-a227-a85ce60604fc None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:18.547487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e8af5-b60c-4631-a227-a85ce60604fc None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:18.547983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927e8af5-b60c-4631-a227-a85ce60604fc None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:18.548929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e8af5-b60c-4631-a227-a85ce60604fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:18.549480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e8af5-b60c-4631-a227-a85ce60604fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:18.551156 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927e8af5-b60c-4631-a227-a85ce60604fc None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:18.554180 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded12f1-5f3a-40f6-a27e-2e9c51254d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:18.554766 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded12f1-5f3a-40f6-a27e-2e9c51254d46 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:18.555050 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded12f1-5f3a-40f6-a27e-2e9c51254d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:18.555326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded12f1-5f3a-40f6-a27e-2e9c51254d46 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:18.556005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded12f1-5f3a-40f6-a27e-2e9c51254d46 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:18.556322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded12f1-5f3a-40f6-a27e-2e9c51254d46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:18.557692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded12f1-5f3a-40f6-a27e-2e9c51254d46 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:18.566802 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 439/865] 10.222.0.45 () {56 vars in 1126 bytes} [Thu Aug 4 02:05:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:18.571007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 427/866] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:18.572153 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded12f1-5f3a-40f6-a27e-2e9c51254d46 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:18.577919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e8af5-b60c-4631-a227-a85ce60604fc None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:18.661519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:18.716132 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927e8af5-b60c-4631-a227-a85ce60604fc None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:18.721592 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:18.730689 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded12f1-5f3a-40f6-a27e-2e9c51254d46 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:18.756530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36397d00-deba-4487-9449-7785856efe48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:18.757206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36397d00-deba-4487-9449-7785856efe48 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:18.757482 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36397d00-deba-4487-9449-7785856efe48 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:18.757753 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36397d00-deba-4487-9449-7785856efe48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:18.766486 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36397d00-deba-4487-9449-7785856efe48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:18.768832 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36397d00-deba-4487-9449-7785856efe48 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:18.778962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853b62c-5645-4910-8655-6142ca611a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:18.779988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853b62c-5645-4910-8655-6142ca611a1e None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:18.780485 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853b62c-5645-4910-8655-6142ca611a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:18.780983 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2853b62c-5645-4910-8655-6142ca611a1e None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:18.781878 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853b62c-5645-4910-8655-6142ca611a1e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:18.782158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-36397d00-deba-4487-9449-7785856efe48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:18.782158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:18.782158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:18.782158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:18.782158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:18.782158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:18.782158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:18.782158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:18.782158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:18.782158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:18.782158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:18.782158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:18.782158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:05:18.782158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:05:18.782158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:05:18.782999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:05:18.782999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:18.782999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:18.782999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:18.782999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:18.782999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:18.782999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:18.782999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:18.782999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:18.782999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:18.782999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:18.782999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:18.782999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:18.782999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:18.782999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:18.782999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:18.782999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:18.788945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:05:18.788945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:05:18.788945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:05:18.788945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:05:18.788945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:05:18.788945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:05:18.788945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:18.788945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:18.788945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 428/867] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:18.789440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853b62c-5645-4910-8655-6142ca611a1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:18.789440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2853b62c-5645-4910-8655-6142ca611a1e None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:18.791115 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36397d00-deba-4487-9449-7785856efe48 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:18.794362 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 440/868] 10.222.0.45 () {56 vars in 1126 bytes} [Thu Aug 4 02:05:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:18.906960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36397d00-deba-4487-9449-7785856efe48 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:18.923399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:18.947535 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad252b5-5466-4b81-a959-b73860fbc486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:18.948161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad252b5-5466-4b81-a959-b73860fbc486 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:18.948438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad252b5-5466-4b81-a959-b73860fbc486 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:18.948741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad252b5-5466-4b81-a959-b73860fbc486 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:18.949495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad252b5-5466-4b81-a959-b73860fbc486 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:18.949798 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad252b5-5466-4b81-a959-b73860fbc486 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:18.951929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad252b5-5466-4b81-a959-b73860fbc486 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:18.964345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 429/869] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:05:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:18.970066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853b62c-5645-4910-8655-6142ca611a1e None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:19.088692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2853b62c-5645-4910-8655-6142ca611a1e None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:19.125076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03656700-62e7-4eeb-9bac-8cfffbf31b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:19.126113 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03656700-62e7-4eeb-9bac-8cfffbf31b74 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:19.126594 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03656700-62e7-4eeb-9bac-8cfffbf31b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:19.127091 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03656700-62e7-4eeb-9bac-8cfffbf31b74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:19.128133 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03656700-62e7-4eeb-9bac-8cfffbf31b74 None admin] RBAC: Adding query filter params (role.id=977824a8c6a24c00af2f6dabd0e2a6f0, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=e54e713b52e84393b018807f8153922e) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:19.128661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03656700-62e7-4eeb-9bac-8cfffbf31b74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:19.130287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03656700-62e7-4eeb-9bac-8cfffbf31b74 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:19.137312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 441/870] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:05:18 2022] GET /identity/v3/role_assignments?user.id=e54e713b52e84393b018807f8153922e&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=977824a8c6a24c00af2f6dabd0e2a6f0 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:19.997007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1beca8-3f69-43ba-b281-423d90b7815f None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:19.997499 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1beca8-3f69-43ba-b281-423d90b7815f None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:19.997499 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1beca8-3f69-43ba-b281-423d90b7815f None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:19.999048 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 430/871] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:20.004516 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462d7cc-8331-4cbc-8027-29790bd23da8 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:20.004980 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462d7cc-8331-4cbc-8027-29790bd23da8 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:20.005185 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462d7cc-8331-4cbc-8027-29790bd23da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:20.024293 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-0462d7cc-8331-4cbc-8027-29790bd23da8 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:20.025543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-0462d7cc-8331-4cbc-8027-29790bd23da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:20.047607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:20.102075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 442/872] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 02:05:20.124118 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1beca8-3f69-43ba-b281-423d90b7815f None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:20.154587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75a5bc-6c97-401f-9baa-fc4f2820f930 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:20.154911 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75a5bc-6c97-401f-9baa-fc4f2820f930 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:20.155138 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75a5bc-6c97-401f-9baa-fc4f2820f930 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:20.156768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 443/873] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:05:20.161672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ce228-dc75-417c-8d9c-2a4d8d5e3c7f None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:20.161953 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ce228-dc75-417c-8d9c-2a4d8d5e3c7f None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:20.162219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ce228-dc75-417c-8d9c-2a4d8d5e3c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:20.181345 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-071ce228-dc75-417c-8d9c-2a4d8d5e3c7f None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:20.182526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-071ce228-dc75-417c-8d9c-2a4d8d5e3c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:20.233283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1beca8-3f69-43ba-b281-423d90b7815f None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:20.271714 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 444/874] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:20.280175 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025de0d-ff44-4deb-be9a-676920376a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:20.281119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025de0d-ff44-4deb-be9a-676920376a5b None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:20.282620 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025de0d-ff44-4deb-be9a-676920376a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:20.282620 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025de0d-ff44-4deb-be9a-676920376a5b None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:20.282985 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025de0d-ff44-4deb-be9a-676920376a5b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:20.284561 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025de0d-ff44-4deb-be9a-676920376a5b None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:20.295179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ce228-dc75-417c-8d9c-2a4d8d5e3c7f None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:20.305317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 431/875] 10.222.0.45 () {60 vars in 1141 bytes} [Thu Aug 4 02:05:20 2022] POST /identity/v3/services => generated 251 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:05:20.395032 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:20.413369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ce228-dc75-417c-8d9c-2a4d8d5e3c7f None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:20.458866 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81b4bc-deec-4e6b-a38e-e382e4cf053b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:20.459514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81b4bc-deec-4e6b-a38e-e382e4cf053b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:20.459791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81b4bc-deec-4e6b-a38e-e382e4cf053b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:20.460062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81b4bc-deec-4e6b-a38e-e382e4cf053b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:20.460712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81b4bc-deec-4e6b-a38e-e382e4cf053b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:20.462087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81b4bc-deec-4e6b-a38e-e382e4cf053b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:20.472256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-4b81b4bc-deec-4e6b-a38e-e382e4cf053b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 02:05:20.472256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:20.472256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:20.472256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:20.472256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:20.472256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:20.472256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:20.472256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:20.472256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 02:05:20.472256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:20.472256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:20.472256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:20.472256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:20.472256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:20.472256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:20.472256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:20.472256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:20.473349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:20.473349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:20.473349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:20.473349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:20.473349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:20.473349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:20.473349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:20.473349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:20.473349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:20.473349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:20.473349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:20.473349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:20.473349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:20.473349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:20.473349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:20.473349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:20.474192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:20.474192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:20.474192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:20.474192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:20.474192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:20.474192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:20.474192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:20.474192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:20.474192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:20.474192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:20.474192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:20.474192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:20.474192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 02:05:20.474192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:20.474192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 445/876] 10.222.0.45 () {56 vars in 1126 bytes} [Thu Aug 4 02:05:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:20.480457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025de0d-ff44-4deb-be9a-676920376a5b None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:20.522241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:20.591971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025de0d-ff44-4deb-be9a-676920376a5b None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:20.627071 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd558f-9180-4a2a-8df9-d9298851d803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:20.627727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd558f-9180-4a2a-8df9-d9298851d803 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:20.628001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd558f-9180-4a2a-8df9-d9298851d803 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:20.628267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd558f-9180-4a2a-8df9-d9298851d803 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:20.628958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd558f-9180-4a2a-8df9-d9298851d803 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:20.629257 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd558f-9180-4a2a-8df9-d9298851d803 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:20.630611 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd558f-9180-4a2a-8df9-d9298851d803 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:20.635817 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 432/877] 10.222.0.45 () {56 vars in 1122 bytes} [Thu Aug 4 02:05:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:20.641348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81b4bc-deec-4e6b-a38e-e382e4cf053b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:20.754653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81b4bc-deec-4e6b-a38e-e382e4cf053b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:20.790801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe41a55-2922-45fb-be3c-d6bc175d9cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:20.791345 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe41a55-2922-45fb-be3c-d6bc175d9cbf None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:20.791629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe41a55-2922-45fb-be3c-d6bc175d9cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:20.791905 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe41a55-2922-45fb-be3c-d6bc175d9cbf None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:20.792591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe41a55-2922-45fb-be3c-d6bc175d9cbf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:20.792927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe41a55-2922-45fb-be3c-d6bc175d9cbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:20.794330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe41a55-2922-45fb-be3c-d6bc175d9cbf None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:20.799416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 446/878] 10.222.0.45 () {56 vars in 1122 bytes} [Thu Aug 4 02:05:20 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:20.931272 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2d24e-bc7e-4434-81cb-0788704b82fb None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:20.931641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2d24e-bc7e-4434-81cb-0788704b82fb None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:20.931815 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2d24e-bc7e-4434-81cb-0788704b82fb None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:20.933531 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 433/879] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:20.939382 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6ccc7-9b85-4652-9b45-591f656aa9aa None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:20.939660 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6ccc7-9b85-4652-9b45-591f656aa9aa None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:20.939926 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf6ccc7-9b85-4652-9b45-591f656aa9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:20.960248 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-ebf6ccc7-9b85-4652-9b45-591f656aa9aa None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:20.961491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-ebf6ccc7-9b85-4652-9b45-591f656aa9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:20.963629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:21.036739 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 447/880] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:20 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:21.059179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2d24e-bc7e-4434-81cb-0788704b82fb None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:21.167576 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:21.173391 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2d24e-bc7e-4434-81cb-0788704b82fb None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:21.208108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21dbfae-19af-4ca6-8ddb-b450440d9ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:21.208759 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dbfae-19af-4ca6-8ddb-b450440d9ef1 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:21.209035 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dbfae-19af-4ca6-8ddb-b450440d9ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:21.209300 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21dbfae-19af-4ca6-8ddb-b450440d9ef1 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:21.209989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dbfae-19af-4ca6-8ddb-b450440d9ef1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:21.211347 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21dbfae-19af-4ca6-8ddb-b450440d9ef1 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:21.232788 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 434/881] 10.222.0.45 () {60 vars in 1141 bytes} [Thu Aug 4 02:05:21 2022] POST /identity/v3/services => generated 269 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:05:21.309988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a167e52-52dc-4c7a-9473-6dbcc9f5a18f None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:21.310184 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a167e52-52dc-4c7a-9473-6dbcc9f5a18f None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:21.310448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a167e52-52dc-4c7a-9473-6dbcc9f5a18f None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:21.312158 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 448/882] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:21.318830 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e174f-5ccd-4897-8571-c916553d6d46 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:21.319156 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e174f-5ccd-4897-8571-c916553d6d46 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:21.319389 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0e174f-5ccd-4897-8571-c916553d6d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:21.338799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-2d0e174f-5ccd-4897-8571-c916553d6d46 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:21.340445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2d0e174f-5ccd-4897-8571-c916553d6d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:21.419233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 435/883] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:21.444194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a167e52-52dc-4c7a-9473-6dbcc9f5a18f None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:21.552798 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a167e52-52dc-4c7a-9473-6dbcc9f5a18f None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:21.592888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07494c0d-a234-4a09-af9f-3f22154d1f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:21.593446 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07494c0d-a234-4a09-af9f-3f22154d1f06 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:21.593722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07494c0d-a234-4a09-af9f-3f22154d1f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:21.593990 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07494c0d-a234-4a09-af9f-3f22154d1f06 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:21.601963 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:21.606134 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07494c0d-a234-4a09-af9f-3f22154d1f06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:05:21.611130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-07494c0d-a234-4a09-af9f-3f22154d1f06 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 02:05:21.611130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:21.611130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:21.611130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:21.611130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:21.611130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:21.611130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:21.611130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:21.611130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:21.611130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:21.611130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:21.611130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:21.611130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:05:21.611130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:05:21.611130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:05:21.611130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:05:21.612612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:05:21.612612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:05:21.612612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:05:21.612612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:05:21.612612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:21.612612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:21.612612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:21.612612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:21.612612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:21.612612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:21.612612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:21.612612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:21.612612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:21.612612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:21.612612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:21.612612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:21.613520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:21.613520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:21.613520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:21.613520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:21.613520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:05:21.613520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:05:21.613520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:05:21.613520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:05:21.613520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:05:21.613520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:05:21.613520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 02:05:21.613520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:21.613520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 449/884] 10.222.0.45 () {56 vars in 1120 bytes} [Thu Aug 4 02:05:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:21.618999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e174f-5ccd-4897-8571-c916553d6d46 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:21.715934 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:21.737005 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0e174f-5ccd-4897-8571-c916553d6d46 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:21.773038 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e47a0-7f90-408b-b6d3-c6bb9fbc9d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:21.773654 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e47a0-7f90-408b-b6d3-c6bb9fbc9d81 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:21.773933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e47a0-7f90-408b-b6d3-c6bb9fbc9d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:21.774205 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e47a0-7f90-408b-b6d3-c6bb9fbc9d81 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:21.774914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86e47a0-7f90-408b-b6d3-c6bb9fbc9d81 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:21.775219 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86e47a0-7f90-408b-b6d3-c6bb9fbc9d81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:21.776578 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86e47a0-7f90-408b-b6d3-c6bb9fbc9d81 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:21.785628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 436/885] 10.222.0.45 () {56 vars in 1115 bytes} [Thu Aug 4 02:05:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:21.793961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07494c0d-a234-4a09-af9f-3f22154d1f06 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:21.900679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07494c0d-a234-4a09-af9f-3f22154d1f06 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:21.936067 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842b3d7-6b72-4828-b19b-2b2b117de343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:21.936067 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842b3d7-6b72-4828-b19b-2b2b117de343 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:21.936067 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842b3d7-6b72-4828-b19b-2b2b117de343 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:21.936067 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a842b3d7-6b72-4828-b19b-2b2b117de343 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:21.948221 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a842b3d7-6b72-4828-b19b-2b2b117de343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:21.949704 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a842b3d7-6b72-4828-b19b-2b2b117de343 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:21.965019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-a842b3d7-6b72-4828-b19b-2b2b117de343 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:21.965019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:21.965019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:21.965019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:21.965019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:21.965019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:21.965019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:21.965019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:21.965019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:21.965019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:21.965019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:21.965019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:21.965019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:21.965019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:21.965019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:21.967095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:21.967095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:21.967095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:21.967095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:21.967095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:21.967095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:21.967095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:21.967095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:21.967095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:21.967095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:21.967095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:21.967095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:21.967095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:21.967095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:21.967095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:21.967095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:21.967095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:21.969329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:21.969329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:21.969329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:21.969329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:21.969329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 450/886] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:21.976412 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e47a0-7f90-408b-b6d3-c6bb9fbc9d81 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:22.084544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e47a0-7f90-408b-b6d3-c6bb9fbc9d81 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:22.120288 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ebb5cc-9401-4eda-b011-a3a168c1d11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:22.121366 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebb5cc-9401-4eda-b011-a3a168c1d11a None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:22.121856 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebb5cc-9401-4eda-b011-a3a168c1d11a None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:22.122332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ebb5cc-9401-4eda-b011-a3a168c1d11a None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:22.123280 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ebb5cc-9401-4eda-b011-a3a168c1d11a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:22.123804 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ebb5cc-9401-4eda-b011-a3a168c1d11a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:22.125404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ebb5cc-9401-4eda-b011-a3a168c1d11a None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:22.138159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 437/887] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:22.144519 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842b3d7-6b72-4828-b19b-2b2b117de343 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:22.200829 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:22.251154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a842b3d7-6b72-4828-b19b-2b2b117de343 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:22.291524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc6718-1f1c-43fb-bf44-6c0004d955c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:22.292166 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc6718-1f1c-43fb-bf44-6c0004d955c6 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:22.292443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc6718-1f1c-43fb-bf44-6c0004d955c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:22.292744 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bc6718-1f1c-43fb-bf44-6c0004d955c6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:22.300610 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc6718-1f1c-43fb-bf44-6c0004d955c6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:22.302060 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bc6718-1f1c-43fb-bf44-6c0004d955c6 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:22.313792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-89bc6718-1f1c-43fb-bf44-6c0004d955c6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 02:05:22.313792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:22.313792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:22.313792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:22.313792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:22.313792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:22.313792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:22.313792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:22.313792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:22.313792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:22.313792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:22.313792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:22.313792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:05:22.313792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:05:22.313792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:05:22.313792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:22.314825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:22.316330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:22.316330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:22.316330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:05:22.316330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:05:22.316330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:05:22.316330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:05:22.316330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:05:22.316330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:05:22.316330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 02:05:22.316330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:22.316330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 451/888] 10.222.0.45 () {56 vars in 1117 bytes} [Thu Aug 4 02:05:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:22.323723 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ebb5cc-9401-4eda-b011-a3a168c1d11a None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:22.372995 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:22.444001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ebb5cc-9401-4eda-b011-a3a168c1d11a None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:22.481267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a9748-7443-4027-a214-834b93792781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:22.481812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a9748-7443-4027-a214-834b93792781 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:22.482092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a9748-7443-4027-a214-834b93792781 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:22.482367 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a9748-7443-4027-a214-834b93792781 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:22.483173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a9748-7443-4027-a214-834b93792781 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:22.483481 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a9748-7443-4027-a214-834b93792781 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:22.484941 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a9748-7443-4027-a214-834b93792781 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:22.503911 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 438/889] 10.222.0.45 () {56 vars in 1149 bytes} [Thu Aug 4 02:05:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:22.509796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc6718-1f1c-43fb-bf44-6c0004d955c6 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:22.582610 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cea25a-ab6f-4247-886f-fd5912c21438 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:22.582923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cea25a-ab6f-4247-886f-fd5912c21438 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:22.583159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cea25a-ab6f-4247-886f-fd5912c21438 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:22.584782 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 439/890] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:22.590670 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b8975-569b-4d8b-8371-821b33a8e28b None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:22.590948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b8975-569b-4d8b-8371-821b33a8e28b None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:22.591216 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b8975-569b-4d8b-8371-821b33a8e28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:22.611029 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-641b8975-569b-4d8b-8371-821b33a8e28b None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:22.612688 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-641b8975-569b-4d8b-8371-821b33a8e28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:22.620810 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bc6718-1f1c-43fb-bf44-6c0004d955c6 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:22.633746 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:22.669823 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87514213-eea2-4616-8f82-beb1b5843721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:22.670462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87514213-eea2-4616-8f82-beb1b5843721 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:22.670743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87514213-eea2-4616-8f82-beb1b5843721 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:22.671017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87514213-eea2-4616-8f82-beb1b5843721 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:22.701591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:22.720668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 440/891] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:22.742190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b8975-569b-4d8b-8371-821b33a8e28b None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:22.752053 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87514213-eea2-4616-8f82-beb1b5843721 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:22.754532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87514213-eea2-4616-8f82-beb1b5843721 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:22.860988 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b8975-569b-4d8b-8371-821b33a8e28b None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:22.883175 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 452/892] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:05:22 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:05:22.889195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87514213-eea2-4616-8f82-beb1b5843721 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:22.900030 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01407b7-ed50-4937-b4d9-66b42ab6534a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:22.900773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01407b7-ed50-4937-b4d9-66b42ab6534a None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:22.901058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01407b7-ed50-4937-b4d9-66b42ab6534a None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:22.901334 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01407b7-ed50-4937-b4d9-66b42ab6534a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:22.902029 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01407b7-ed50-4937-b4d9-66b42ab6534a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:22.902534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:22.903510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01407b7-ed50-4937-b4d9-66b42ab6534a None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:22.913626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-b01407b7-ed50-4937-b4d9-66b42ab6534a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 02:05:22.913626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:22.913626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:22.913626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:22.913626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:22.913626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:22.913626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:22.913626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:22.913626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 02:05:22.913626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:22.913626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:22.913626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:22.913626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:22.913626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:22.913626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:22.913626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:22.913626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:22.914634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:22.914634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:22.914634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:22.914634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:22.914634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:22.914634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:22.914634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:22.914634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:22.914634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:22.914634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:22.914634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:22.914634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:22.914634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:22.914634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:22.914634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:22.914634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:22.921106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:22.921106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:22.921106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:22.921106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:22.921106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:22.921106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:22.921106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:22.921106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:22.921106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:22.921106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:22.921106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:22.921106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:22.921106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 02:05:22.921106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:22.921106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 441/893] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:05:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:22.923973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01407b7-ed50-4937-b4d9-66b42ab6534a None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:22.991314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:23.008486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87514213-eea2-4616-8f82-beb1b5843721 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:23.036217 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01407b7-ed50-4937-b4d9-66b42ab6534a None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:23.044694 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7160a73-5e52-44e3-96c0-f3e1118bfbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:23.045337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7160a73-5e52-44e3-96c0-f3e1118bfbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:23.045623 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7160a73-5e52-44e3-96c0-f3e1118bfbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:23.045889 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7160a73-5e52-44e3-96c0-f3e1118bfbc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:23.057194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7160a73-5e52-44e3-96c0-f3e1118bfbc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:23.058632 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7160a73-5e52-44e3-96c0-f3e1118bfbc9 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:23.072202 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542a630-680e-4cad-a3b5-12c2c14ade79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:23.072848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542a630-680e-4cad-a3b5-12c2c14ade79 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:23.073124 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542a630-680e-4cad-a3b5-12c2c14ade79 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:23.073390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542a630-680e-4cad-a3b5-12c2c14ade79 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:23.074060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542a630-680e-4cad-a3b5-12c2c14ade79 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:23.074357 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542a630-680e-4cad-a3b5-12c2c14ade79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:23.074667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-c7160a73-5e52-44e3-96c0-f3e1118bfbc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:23.074667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:23.074667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:23.074667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:23.074667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:23.074667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:23.074667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:23.074667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:23.074667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:23.074667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:23.074667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:23.074667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:23.074667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:23.074667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:23.074667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:23.075500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:23.075500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:23.075500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:23.075500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:23.075500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:23.075500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:23.075500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:23.075500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:23.075500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:23.075500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:23.075500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:23.075500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:23.075500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:23.075500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:23.075500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:23.075500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:23.075500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:23.076389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:23.076389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:23.076389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:23.076389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:23.076607 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542a630-680e-4cad-a3b5-12c2c14ade79 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:23.076710 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 453/894] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:23.077708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5ef9b-d9c9-4730-937a-75e2d1156218 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:23.077991 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5ef9b-d9c9-4730-937a-75e2d1156218 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:23.078265 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5ef9b-d9c9-4730-937a-75e2d1156218 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:23.083488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 454/895] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:23.083488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5ef9b-d9c9-4730-937a-75e2d1156218 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:23.085964 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 442/896] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:05:22 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:23.091847 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dcf00-d6b6-431a-b56a-50a49516ff08 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:23.091847 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dcf00-d6b6-431a-b56a-50a49516ff08 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:23.091847 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dcf00-d6b6-431a-b56a-50a49516ff08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:23.105645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-767dcf00-d6b6-431a-b56a-50a49516ff08 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:23.106812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-767dcf00-d6b6-431a-b56a-50a49516ff08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:23.166018 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:23.180658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 443/897] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:23.180658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767dcf00-d6b6-431a-b56a-50a49516ff08 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:23.228846 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:23.229504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5ef9b-d9c9-4730-937a-75e2d1156218 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:23.266422 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c269e93-9f45-481b-86df-2815005f1fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:23.266952 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c269e93-9f45-481b-86df-2815005f1fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:23.267230 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c269e93-9f45-481b-86df-2815005f1fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:23.267508 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c269e93-9f45-481b-86df-2815005f1fb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:23.268219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c269e93-9f45-481b-86df-2815005f1fb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:23.268533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c269e93-9f45-481b-86df-2815005f1fb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:23.269950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c269e93-9f45-481b-86df-2815005f1fb0 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:23.280849 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767dcf00-d6b6-431a-b56a-50a49516ff08 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:23.283049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 455/898] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:23.284508 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c269e93-9f45-481b-86df-2815005f1fb0 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:23.325432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03df300-babd-424b-9be5-978f6b2f467f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:23.326052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03df300-babd-424b-9be5-978f6b2f467f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:23.326326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03df300-babd-424b-9be5-978f6b2f467f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:23.326596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03df300-babd-424b-9be5-978f6b2f467f None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:23.327263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03df300-babd-424b-9be5-978f6b2f467f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:23.327561 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03df300-babd-424b-9be5-978f6b2f467f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:23.328989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03df300-babd-424b-9be5-978f6b2f467f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:23.334257 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 444/899] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:05:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:23.335068 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03df300-babd-424b-9be5-978f6b2f467f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:23.398894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c269e93-9f45-481b-86df-2815005f1fb0 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:23.421376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:23.434012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4fe5e-f980-4756-8ded-74e99d161632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:23.434623 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4fe5e-f980-4756-8ded-74e99d161632 None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:23.434898 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4fe5e-f980-4756-8ded-74e99d161632 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:23.435170 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4fe5e-f980-4756-8ded-74e99d161632 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:23.435846 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4fe5e-f980-4756-8ded-74e99d161632 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:23.437237 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4fe5e-f980-4756-8ded-74e99d161632 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:23.438368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03df300-babd-424b-9be5-978f6b2f467f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:23.451735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 456/900] 10.222.0.45 () {60 vars in 1141 bytes} [Thu Aug 4 02:05:23 2022] POST /identity/v3/services => generated 241 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:05:23.452462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4fe5e-f980-4756-8ded-74e99d161632 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:23.465769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:23.485288 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731437a-6a18-47a9-8f30-bb53120266ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:23.485951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731437a-6a18-47a9-8f30-bb53120266ad None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:23.486232 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731437a-6a18-47a9-8f30-bb53120266ad None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:23.486510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731437a-6a18-47a9-8f30-bb53120266ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:23.493944 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731437a-6a18-47a9-8f30-bb53120266ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:23.495419 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731437a-6a18-47a9-8f30-bb53120266ad None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:23.507555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-1731437a-6a18-47a9-8f30-bb53120266ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:23.507555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:23.507555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:23.507555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:23.507555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:23.507555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:23.507555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:23.507555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:23.507555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:23.507555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:23.507555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:23.507555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:23.507555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:05:23.507555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:05:23.507555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:05:23.508721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:05:23.508721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:23.508721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:23.508721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:23.508721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:23.508721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:23.508721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:23.508721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:23.508721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:23.508721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:23.508721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:23.508721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:23.508721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:23.508721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:23.508721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:23.508721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:23.508721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:23.509663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:05:23.509663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:05:23.509663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:05:23.509663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:05:23.509663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:05:23.509663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:05:23.509663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:23.509663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:23.509663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 445/901] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:23.510297 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aca80b-7d5c-4421-82d7-24ebd87c4b16 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:23.510422 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aca80b-7d5c-4421-82d7-24ebd87c4b16 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:23.510698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aca80b-7d5c-4421-82d7-24ebd87c4b16 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:23.516143 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 446/902] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:23.517147 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aca80b-7d5c-4421-82d7-24ebd87c4b16 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:23.590146 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4fe5e-f980-4756-8ded-74e99d161632 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:23.628891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac23f0a-d928-4c8b-854f-fdcb2ffec55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:23.629516 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac23f0a-d928-4c8b-854f-fdcb2ffec55d None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:23.629795 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac23f0a-d928-4c8b-854f-fdcb2ffec55d None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:23.630067 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac23f0a-d928-4c8b-854f-fdcb2ffec55d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:23.631704 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac23f0a-d928-4c8b-854f-fdcb2ffec55d None admin] RBAC: Adding query filter params (interface=public, service_id=0bb574d93aab4264b0de42d24977d51c, region_id=RegionOne) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:23.631704 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac23f0a-d928-4c8b-854f-fdcb2ffec55d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:23.632490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac23f0a-d928-4c8b-854f-fdcb2ffec55d None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:23.637252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aca80b-7d5c-4421-82d7-24ebd87c4b16 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:23.639356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 457/903] 10.222.0.45 () {56 vars in 1251 bytes} [Thu Aug 4 02:05:23 2022] GET /identity/v3/endpoints?service_id=0bb574d93aab4264b0de42d24977d51c&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:23.640657 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a76b8-a4eb-4e59-bc59-b86b4c21222c None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:23.646421 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a76b8-a4eb-4e59-bc59-b86b4c21222c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:23.646706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a76b8-a4eb-4e59-bc59-b86b4c21222c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:23.666086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-9f8a76b8-a4eb-4e59-bc59-b86b4c21222c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:23.667291 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9f8a76b8-a4eb-4e59-bc59-b86b4c21222c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:23.686308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d7707-642f-447a-b9bb-e43a832cef55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:23.686975 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d7707-642f-447a-b9bb-e43a832cef55 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:23.687320 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d7707-642f-447a-b9bb-e43a832cef55 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:23.687594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d7707-642f-447a-b9bb-e43a832cef55 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:23.689130 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d7707-642f-447a-b9bb-e43a832cef55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:23.689470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d7707-642f-447a-b9bb-e43a832cef55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:23.691471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d7707-642f-447a-b9bb-e43a832cef55 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:23.707018 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 447/904] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:05:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:23.707866 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d7707-642f-447a-b9bb-e43a832cef55 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:23.715019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:23.744186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:23.748662 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 458/905] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:23.749546 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a76b8-a4eb-4e59-bc59-b86b4c21222c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:23.835264 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d7707-642f-447a-b9bb-e43a832cef55 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:23.861313 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a76b8-a4eb-4e59-bc59-b86b4c21222c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:23.870556 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e83358-08a3-4cb2-a736-b55f6eb6bddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:23.871181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e83358-08a3-4cb2-a736-b55f6eb6bddb None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:23.871463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e83358-08a3-4cb2-a736-b55f6eb6bddb None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:23.871735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e83358-08a3-4cb2-a736-b55f6eb6bddb None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:23.872389 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e83358-08a3-4cb2-a736-b55f6eb6bddb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:23.873807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e83358-08a3-4cb2-a736-b55f6eb6bddb None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:23.880531 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 448/906] 10.222.0.45 () {56 vars in 1087 bytes} [Thu Aug 4 02:05:23 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:05:23.880531 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e83358-08a3-4cb2-a736-b55f6eb6bddb None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:23.906408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074482db-209a-4e72-b84b-a873499b1c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:23.907356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074482db-209a-4e72-b84b-a873499b1c55 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:23.907801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074482db-209a-4e72-b84b-a873499b1c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:23.908072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074482db-209a-4e72-b84b-a873499b1c55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:23.909150 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074482db-209a-4e72-b84b-a873499b1c55 None admin] RBAC: Adding query filter params (role.id=609fcaecc1a648f8b41c081fe0703ef4, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=e54e713b52e84393b018807f8153922e) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:23.909466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074482db-209a-4e72-b84b-a873499b1c55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:23.910965 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074482db-209a-4e72-b84b-a873499b1c55 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:23.924675 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 459/907] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:05:23 2022] GET /identity/v3/role_assignments?user.id=e54e713b52e84393b018807f8153922e&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=609fcaecc1a648f8b41c081fe0703ef4 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:24.019049 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:24.026602 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e83358-08a3-4cb2-a736-b55f6eb6bddb None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:24.073257 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c267c3-2abc-479f-a787-dd1270bf00bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:24.073934 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c267c3-2abc-479f-a787-dd1270bf00bc None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:24.074274 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c267c3-2abc-479f-a787-dd1270bf00bc None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:24.074574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c267c3-2abc-479f-a787-dd1270bf00bc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:24.075299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c267c3-2abc-479f-a787-dd1270bf00bc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:24.076800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c267c3-2abc-479f-a787-dd1270bf00bc None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:24.089634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-70c267c3-2abc-479f-a787-dd1270bf00bc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 02:05:24.089634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:24.089634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:24.089634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:24.089634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:24.089634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:24.089634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:24.089634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:24.089634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 02:05:24.089634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:24.089634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:24.089634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:24.089634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:24.089634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:24.089634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:24.089634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:24.091054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:24.091054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:24.091054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:24.091054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:24.091054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:24.091054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:24.091054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:24.091054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:24.091054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:24.091054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:24.091054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:24.091054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:24.091054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:24.091054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:24.091054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:24.091054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:24.091912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:24.091912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:24.091912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:24.091912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:24.091912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:24.091912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:24.091912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:24.091912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:24.091912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:24.091912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:24.091912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:24.091912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:24.091912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:24.091912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 02:05:24.091912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:24.091912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 449/908] 10.222.0.45 () {56 vars in 1132 bytes} [Thu Aug 4 02:05:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:24.100595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074482db-209a-4e72-b84b-a873499b1c55 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:24.149473 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:24.213042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074482db-209a-4e72-b84b-a873499b1c55 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:24.253209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a03cb-5624-42d7-b379-1a3de4cecbff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:24.253785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a03cb-5624-42d7-b379-1a3de4cecbff None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:24.254066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a03cb-5624-42d7-b379-1a3de4cecbff None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:24.254340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a03cb-5624-42d7-b379-1a3de4cecbff None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:24.255096 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a03cb-5624-42d7-b379-1a3de4cecbff None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:24.255404 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a03cb-5624-42d7-b379-1a3de4cecbff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:24.256806 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a03cb-5624-42d7-b379-1a3de4cecbff None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:24.262523 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 460/909] 10.222.0.45 () {56 vars in 1126 bytes} [Thu Aug 4 02:05:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:24.268127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c267c3-2abc-479f-a787-dd1270bf00bc None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:24.375833 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c267c3-2abc-479f-a787-dd1270bf00bc None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:24.420312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78725dbc-6416-4975-9098-31db0b0953ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:24.420958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78725dbc-6416-4975-9098-31db0b0953ce None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:24.421239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78725dbc-6416-4975-9098-31db0b0953ce None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:24.421510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78725dbc-6416-4975-9098-31db0b0953ce None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:24.422192 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78725dbc-6416-4975-9098-31db0b0953ce None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:24.422495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78725dbc-6416-4975-9098-31db0b0953ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:24.423861 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78725dbc-6416-4975-9098-31db0b0953ce None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:24.429481 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 450/910] 10.222.0.45 () {56 vars in 1126 bytes} [Thu Aug 4 02:05:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:24.435313 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a03cb-5624-42d7-b379-1a3de4cecbff None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:24.555201 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a03cb-5624-42d7-b379-1a3de4cecbff None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:24.558009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:24.595261 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fbc47e-6d27-4e67-8f7d-e3d2f6e8add6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:24.595879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbc47e-6d27-4e67-8f7d-e3d2f6e8add6 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:24.596157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbc47e-6d27-4e67-8f7d-e3d2f6e8add6 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:24.596981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fbc47e-6d27-4e67-8f7d-e3d2f6e8add6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:24.597742 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbc47e-6d27-4e67-8f7d-e3d2f6e8add6 None admin] RBAC: Adding query filter params (interface=public, service_id=2050b242e2e94869955bd996b9f34769, region_id=RegionOne) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:24.598048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbc47e-6d27-4e67-8f7d-e3d2f6e8add6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:24.599408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fbc47e-6d27-4e67-8f7d-e3d2f6e8add6 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:24.605471 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 461/911] 10.222.0.45 () {56 vars in 1251 bytes} [Thu Aug 4 02:05:24 2022] GET /identity/v3/endpoints?service_id=2050b242e2e94869955bd996b9f34769&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:24.610993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78725dbc-6416-4975-9098-31db0b0953ce None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:24.639773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:24.737939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78725dbc-6416-4975-9098-31db0b0953ce None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:24.780078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dd4436-3969-466e-8349-71dad56e72c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:24.781539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd4436-3969-466e-8349-71dad56e72c7 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:24.782033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd4436-3969-466e-8349-71dad56e72c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:24.782530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd4436-3969-466e-8349-71dad56e72c7 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:24.783414 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd4436-3969-466e-8349-71dad56e72c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:24.785610 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dd4436-3969-466e-8349-71dad56e72c7 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:24.793139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 451/912] 10.222.0.45 () {56 vars in 1087 bytes} [Thu Aug 4 02:05:24 2022] GET /identity/v3/services => generated 1497 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:05:25.592385 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ff971-bf3d-4508-92d3-0b4bb6dbde96 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:25.592934 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ff971-bf3d-4508-92d3-0b4bb6dbde96 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:25.592934 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ff971-bf3d-4508-92d3-0b4bb6dbde96 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:25.594531 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 462/913] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:25.600596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda3f10-b150-410d-93b3-43a47b0c0399 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:25.601081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda3f10-b150-410d-93b3-43a47b0c0399 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:25.601157 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dda3f10-b150-410d-93b3-43a47b0c0399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:25.619987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-0dda3f10-b150-410d-93b3-43a47b0c0399 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:25.621492 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-0dda3f10-b150-410d-93b3-43a47b0c0399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:25.698104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 452/914] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:25.720960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ff971-bf3d-4508-92d3-0b4bb6dbde96 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:25.831653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ff971-bf3d-4508-92d3-0b4bb6dbde96 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:25.871575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db9a58-7d1c-4a9f-9ec7-50ca90e888d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:25.872240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db9a58-7d1c-4a9f-9ec7-50ca90e888d7 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:25.872518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db9a58-7d1c-4a9f-9ec7-50ca90e888d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:25.872823 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8db9a58-7d1c-4a9f-9ec7-50ca90e888d7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:25.873462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db9a58-7d1c-4a9f-9ec7-50ca90e888d7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:25.874870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8db9a58-7d1c-4a9f-9ec7-50ca90e888d7 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:25.885266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-e8db9a58-7d1c-4a9f-9ec7-50ca90e888d7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 02:05:25.885266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:25.885266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:25.885266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:25.885266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:25.885266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:25.885266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:25.885266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:25.885266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 02:05:25.885266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:25.885266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:25.885266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:25.885266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:25.885266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:25.885266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:25.885266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:25.885266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:25.886322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:25.886322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:25.886322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:25.886322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:25.886322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:25.886322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:25.886322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:25.886322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:25.886322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:25.886322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:25.886322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:25.886322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:25.886322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:25.886322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:25.886322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:25.886322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:25.887179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:25.887179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:25.887179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:25.887179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:25.887179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:25.887179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:25.887179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:25.887179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:25.887179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:25.887179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:25.887179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:25.887179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:25.887179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 02:05:25.887179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:25.887179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 463/915] 10.222.0.45 () {56 vars in 1126 bytes} [Thu Aug 4 02:05:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:25.893773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda3f10-b150-410d-93b3-43a47b0c0399 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:25.917964 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:26.018012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dda3f10-b150-410d-93b3-43a47b0c0399 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:26.053825 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a293916-c5d0-4292-a67e-770bab95bafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:26.054447 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a293916-c5d0-4292-a67e-770bab95bafb None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:26.054720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a293916-c5d0-4292-a67e-770bab95bafb None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:26.054987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a293916-c5d0-4292-a67e-770bab95bafb None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:26.055661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a293916-c5d0-4292-a67e-770bab95bafb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:26.055959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a293916-c5d0-4292-a67e-770bab95bafb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:26.057346 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a293916-c5d0-4292-a67e-770bab95bafb None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:26.062754 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 453/916] 10.222.0.45 () {56 vars in 1122 bytes} [Thu Aug 4 02:05:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:26.069317 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db9a58-7d1c-4a9f-9ec7-50ca90e888d7 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:26.098495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ca890-25ac-4d0b-b888-6d2273754946 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:26.098813 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ca890-25ac-4d0b-b888-6d2273754946 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:26.099028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ca890-25ac-4d0b-b888-6d2273754946 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:26.100650 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 454/917] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:05:26.106583 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:26.114034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ad3d2-7526-4bfb-a4e8-f287cf0a70a0 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:26.114320 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ad3d2-7526-4bfb-a4e8-f287cf0a70a0 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:26.114578 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ad3d2-7526-4bfb-a4e8-f287cf0a70a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:26.135864 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-fd2ad3d2-7526-4bfb-a4e8-f287cf0a70a0 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:26.137675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-fd2ad3d2-7526-4bfb-a4e8-f287cf0a70a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:26.184438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8db9a58-7d1c-4a9f-9ec7-50ca90e888d7 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:26.217554 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 455/918] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 02:05:26.220842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376407ca-ce72-4461-ac9b-f7d67498edea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:26.221086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376407ca-ce72-4461-ac9b-f7d67498edea None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:26.221305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376407ca-ce72-4461-ac9b-f7d67498edea None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:26.221582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376407ca-ce72-4461-ac9b-f7d67498edea None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:26.222255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376407ca-ce72-4461-ac9b-f7d67498edea None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:26.222554 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376407ca-ce72-4461-ac9b-f7d67498edea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:26.223959 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376407ca-ce72-4461-ac9b-f7d67498edea None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:26.227983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbb0be-42c8-4562-b321-443633c146f9 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:26.227983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbb0be-42c8-4562-b321-443633c146f9 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:26.227983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbb0be-42c8-4562-b321-443633c146f9 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:26.227983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 456/919] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:26.233433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 464/920] 10.222.0.45 () {56 vars in 1122 bytes} [Thu Aug 4 02:05:26 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:26.233889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee575872-dac5-42d5-99f8-dcb8d70d35dd None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:26.234171 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee575872-dac5-42d5-99f8-dcb8d70d35dd None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:26.234442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee575872-dac5-42d5-99f8-dcb8d70d35dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:26.242520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376407ca-ce72-4461-ac9b-f7d67498edea None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:26.257256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-ee575872-dac5-42d5-99f8-dcb8d70d35dd None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:26.258450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ee575872-dac5-42d5-99f8-dcb8d70d35dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:26.266554 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:26.336968 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 457/921] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:26.338743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee575872-dac5-42d5-99f8-dcb8d70d35dd None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:26.345201 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:26.351619 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376407ca-ce72-4461-ac9b-f7d67498edea None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:26.389430 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ee588-e712-41f7-b671-85ff79bbde04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:26.390313 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ee588-e712-41f7-b671-85ff79bbde04 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:26.390593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ee588-e712-41f7-b671-85ff79bbde04 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:26.391161 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ee588-e712-41f7-b671-85ff79bbde04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:26.391986 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ee588-e712-41f7-b671-85ff79bbde04 None admin] RBAC: Adding query filter params (interface=public, service_id=184a26d33f4c4d00851117a1c90761b9, region_id=RegionOne) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:26.392230 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ee588-e712-41f7-b671-85ff79bbde04 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:26.393662 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ee588-e712-41f7-b671-85ff79bbde04 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:26.403455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 465/922] 10.222.0.45 () {56 vars in 1251 bytes} [Thu Aug 4 02:05:26 2022] GET /identity/v3/endpoints?service_id=184a26d33f4c4d00851117a1c90761b9&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:26.404221 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ee588-e712-41f7-b671-85ff79bbde04 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:26.452228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee575872-dac5-42d5-99f8-dcb8d70d35dd None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:26.487875 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01a7ef-0c32-43d2-83c6-64c29ba8d266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:26.488526 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01a7ef-0c32-43d2-83c6-64c29ba8d266 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:26.488832 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01a7ef-0c32-43d2-83c6-64c29ba8d266 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:26.489104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf01a7ef-0c32-43d2-83c6-64c29ba8d266 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:26.489732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf01a7ef-0c32-43d2-83c6-64c29ba8d266 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:26.491094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf01a7ef-0c32-43d2-83c6-64c29ba8d266 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:26.500064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-bf01a7ef-0c32-43d2-83c6-64c29ba8d266 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 02:05:26.500064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:26.500064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:26.500064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:26.500064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:26.500064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:26.500064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:26.500064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:26.500064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 02:05:26.500064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:26.500064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:26.500064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:26.500064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:26.500064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:26.500064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:26.500064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:26.500064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:26.501188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:26.501188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:26.501188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:26.501188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:26.501188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:26.501188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:26.501188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:26.501188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:26.501188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:26.501188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:26.501188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:26.501188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:26.501188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:26.501188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:26.501188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:26.501188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:26.502026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:26.502026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:26.502026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:26.502026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:26.502026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:26.502026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:26.502026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:26.502026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:26.502026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:26.502026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:26.502026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:26.502026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:26.502026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 04 02:05:26.502026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:26.502026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 458/923] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:05:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:26.503122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01a7ef-0c32-43d2-83c6-64c29ba8d266 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:26.513417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:26.522088 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ee588-e712-41f7-b671-85ff79bbde04 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:26.562568 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49633a66-5d5a-4cab-821e-10ca9fd5bca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:26.563760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49633a66-5d5a-4cab-821e-10ca9fd5bca1 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:26.564199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49633a66-5d5a-4cab-821e-10ca9fd5bca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:26.564473 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49633a66-5d5a-4cab-821e-10ca9fd5bca1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:26.576347 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49633a66-5d5a-4cab-821e-10ca9fd5bca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:05:26.581320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-49633a66-5d5a-4cab-821e-10ca9fd5bca1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 02:05:26.581320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:26.581320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:26.581320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:26.581320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:26.581320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:26.581320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:26.581320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:26.581320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:26.581320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:26.581320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:26.581320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:26.581320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:05:26.581320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:05:26.581320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:05:26.581320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:05:26.582336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:05:26.582336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:05:26.582336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:05:26.582336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:05:26.582336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:26.582336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:26.582336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:26.582336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:26.582336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:26.582336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:26.582336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:26.582336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:26.582336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:26.582336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:26.582336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:26.582336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:26.583188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:26.583188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:26.583188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:26.583188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:26.583188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:05:26.583188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:05:26.583188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:05:26.583188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:05:26.583188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:05:26.583188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:05:26.583188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 02:05:26.583188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:26.587541 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 466/924] 10.222.0.45 () {56 vars in 1120 bytes} [Thu Aug 4 02:05:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:26.588420 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49633a66-5d5a-4cab-821e-10ca9fd5bca1 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:26.611611 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:26.612764 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf01a7ef-0c32-43d2-83c6-64c29ba8d266 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:26.648087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02ab75-8806-4d29-a14c-f4adb5d88ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:26.648644 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02ab75-8806-4d29-a14c-f4adb5d88ecd None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:26.648926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02ab75-8806-4d29-a14c-f4adb5d88ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:26.649197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02ab75-8806-4d29-a14c-f4adb5d88ecd None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:26.649847 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02ab75-8806-4d29-a14c-f4adb5d88ecd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:26.651222 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c02ab75-8806-4d29-a14c-f4adb5d88ecd None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:26.667046 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 459/925] 10.222.0.45 () {56 vars in 1087 bytes} [Thu Aug 4 02:05:26 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:05:26.667046 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02ab75-8806-4d29-a14c-f4adb5d88ecd None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:26.730271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49633a66-5d5a-4cab-821e-10ca9fd5bca1 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:26.763064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:26.777887 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5cede-294b-4062-8bd7-24bae3c3cdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:26.778765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5cede-294b-4062-8bd7-24bae3c3cdc6 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:26.779114 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5cede-294b-4062-8bd7-24bae3c3cdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:26.779454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a5cede-294b-4062-8bd7-24bae3c3cdc6 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:26.780339 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5cede-294b-4062-8bd7-24bae3c3cdc6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:26.780791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5cede-294b-4062-8bd7-24bae3c3cdc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:26.782291 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a5cede-294b-4062-8bd7-24bae3c3cdc6 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:26.793779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 467/926] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:05:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:26.800522 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5cede-294b-4062-8bd7-24bae3c3cdc6 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:26.805858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02ab75-8806-4d29-a14c-f4adb5d88ecd None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:26.849179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0d674-ceb7-4089-a1a7-664ba1d23a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:26.849814 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0d674-ceb7-4089-a1a7-664ba1d23a82 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:26.850090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0d674-ceb7-4089-a1a7-664ba1d23a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:26.850382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a0d674-ceb7-4089-a1a7-664ba1d23a82 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:26.851104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0d674-ceb7-4089-a1a7-664ba1d23a82 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:26.851402 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0d674-ceb7-4089-a1a7-664ba1d23a82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:26.852808 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a0d674-ceb7-4089-a1a7-664ba1d23a82 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:26.864677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 460/927] 10.222.0.45 () {56 vars in 1115 bytes} [Thu Aug 4 02:05:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:26.870799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0d674-ceb7-4089-a1a7-664ba1d23a82 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:26.916103 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:26.932565 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a5cede-294b-4062-8bd7-24bae3c3cdc6 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:26.970530 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a267d37e-1c87-421b-b911-146656a557ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:26.971512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267d37e-1c87-421b-b911-146656a557ac None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:26.971798 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267d37e-1c87-421b-b911-146656a557ac None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:26.972080 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a267d37e-1c87-421b-b911-146656a557ac None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:26.972799 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a267d37e-1c87-421b-b911-146656a557ac None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:26.973109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a267d37e-1c87-421b-b911-146656a557ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:26.975170 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a267d37e-1c87-421b-b911-146656a557ac None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:26.988663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 468/928] 10.222.0.45 () {56 vars in 1114 bytes} [Thu Aug 4 02:05:26 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:26.988663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735538c-14bb-4f50-9f21-a3d52b00136b None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:26.988663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735538c-14bb-4f50-9f21-a3d52b00136b None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:26.988663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735538c-14bb-4f50-9f21-a3d52b00136b None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:26.988663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 469/929] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:26.989126 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a0d674-ceb7-4089-a1a7-664ba1d23a82 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:26.989203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735538c-14bb-4f50-9f21-a3d52b00136b None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:27.017161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:27.023931 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739f939-4c16-433d-b3cc-d95416dbf92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:27.024688 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739f939-4c16-433d-b3cc-d95416dbf92d None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:27.025047 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739f939-4c16-433d-b3cc-d95416dbf92d None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:27.025335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739f939-4c16-433d-b3cc-d95416dbf92d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:27.037229 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e739f939-4c16-433d-b3cc-d95416dbf92d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:27.038695 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e739f939-4c16-433d-b3cc-d95416dbf92d None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:27.053519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-e739f939-4c16-433d-b3cc-d95416dbf92d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:27.053519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:27.053519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:27.053519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:27.053519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:27.053519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:27.053519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:27.053519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:27.053519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:27.053519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:27.053519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:27.053519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:27.053519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:27.053519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:27.053519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:27.054628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:27.054628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:27.054628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:27.054628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:27.054628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:27.054628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:27.054628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:27.054628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:27.054628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:27.054628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:27.054628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:27.054628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:27.054628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:27.054628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:27.054628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:27.054628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:27.054628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:27.055544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:27.055544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:27.055544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:27.055544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:27.055544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 461/930] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:27.056245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311a840-d19d-405f-872b-43565c682fdb None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:27.056245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311a840-d19d-405f-872b-43565c682fdb None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:27.056415 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311a840-d19d-405f-872b-43565c682fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:27.075252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-0311a840-d19d-405f-872b-43565c682fdb None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:27.076452 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-0311a840-d19d-405f-872b-43565c682fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:27.107198 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735538c-14bb-4f50-9f21-a3d52b00136b None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:27.144544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ce238-fdb8-43df-b835-94d09d042959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:27.145544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ce238-fdb8-43df-b835-94d09d042959 None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:27.145744 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 462/931] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:27.146236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ce238-fdb8-43df-b835-94d09d042959 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:27.146590 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311a840-d19d-405f-872b-43565c682fdb None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:27.147337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7ce238-fdb8-43df-b835-94d09d042959 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:27.148765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ce238-fdb8-43df-b835-94d09d042959 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:27.151438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7ce238-fdb8-43df-b835-94d09d042959 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:27.179541 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 470/932] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:05:26 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:05:27.180396 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ce238-fdb8-43df-b835-94d09d042959 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:27.201059 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:27.294473 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311a840-d19d-405f-872b-43565c682fdb None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:27.313136 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:27.337745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7ce238-fdb8-43df-b835-94d09d042959 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:27.338700 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1baba4-7527-4202-89a7-bbdc6f46526f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:27.339707 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1baba4-7527-4202-89a7-bbdc6f46526f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:27.340816 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1baba4-7527-4202-89a7-bbdc6f46526f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:27.341097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1baba4-7527-4202-89a7-bbdc6f46526f None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:27.341851 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1baba4-7527-4202-89a7-bbdc6f46526f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:27.342206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1baba4-7527-4202-89a7-bbdc6f46526f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:27.343709 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1baba4-7527-4202-89a7-bbdc6f46526f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:27.358311 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 463/933] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:27.365008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1baba4-7527-4202-89a7-bbdc6f46526f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:27.383977 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7fad8-9306-46b0-863a-85dc247f20a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:27.384630 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7fad8-9306-46b0-863a-85dc247f20a7 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:27.384924 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7fad8-9306-46b0-863a-85dc247f20a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:27.385197 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be7fad8-9306-46b0-863a-85dc247f20a7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:27.385828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7fad8-9306-46b0-863a-85dc247f20a7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:27.387196 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be7fad8-9306-46b0-863a-85dc247f20a7 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:27.397377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-0be7fad8-9306-46b0-863a-85dc247f20a7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 02:05:27.397377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:27.397377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:27.397377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:27.397377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:27.397377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:27.397377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:27.397377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:27.397377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 02:05:27.397377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:27.397377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:27.397377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:27.397377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:27.397377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:27.397377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:27.397377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:27.398438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:27.398438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:27.398438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:27.398438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:27.398438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:27.398438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:27.398438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:27.398438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:27.398438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:27.398438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:27.398438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:27.398438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:27.398438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:27.398438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:27.398438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:27.398438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:27.399310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:27.399310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:27.399310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:27.399310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:27.399310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:27.399310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:27.399310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:27.399310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:27.399310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:27.399310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:27.399310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:27.399310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:27.399310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:27.399310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 04 02:05:27.399310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:27.399310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 471/934] 10.222.0.45 () {56 vars in 1132 bytes} [Thu Aug 4 02:05:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:27.404442 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7fad8-9306-46b0-863a-85dc247f20a7 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:27.487535 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:27.495741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1baba4-7527-4202-89a7-bbdc6f46526f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:27.533067 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be7fad8-9306-46b0-863a-85dc247f20a7 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:27.536708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a839a-83dd-4df5-8ed0-d2ea5e40a908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:27.537368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a839a-83dd-4df5-8ed0-d2ea5e40a908 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:27.537650 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a839a-83dd-4df5-8ed0-d2ea5e40a908 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:27.537922 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a839a-83dd-4df5-8ed0-d2ea5e40a908 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:27.546388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a839a-83dd-4df5-8ed0-d2ea5e40a908 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:27.547969 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a839a-83dd-4df5-8ed0-d2ea5e40a908 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:27.561080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-090a839a-83dd-4df5-8ed0-d2ea5e40a908 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 02:05:27.561080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:27.561080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:27.561080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:27.561080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:27.561080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:27.561080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:27.561080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:27.561080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:27.561080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:27.561080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:27.561080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:27.561080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:05:27.561080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:05:27.561080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:05:27.561080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:27.562052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:27.563252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:27.563252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:27.563252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:05:27.563252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:05:27.563252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:05:27.563252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:05:27.563252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:05:27.563252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:05:27.563252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 02:05:27.563252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:27.563252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 464/935] 10.222.0.45 () {56 vars in 1117 bytes} [Thu Aug 4 02:05:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:27.569448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a839a-83dd-4df5-8ed0-d2ea5e40a908 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:27.581111 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476ad94-39aa-4afc-b3c6-1b6d31cc4282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:27.582510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476ad94-39aa-4afc-b3c6-1b6d31cc4282 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:27.583423 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476ad94-39aa-4afc-b3c6-1b6d31cc4282 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:27.584060 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5476ad94-39aa-4afc-b3c6-1b6d31cc4282 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:27.585083 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476ad94-39aa-4afc-b3c6-1b6d31cc4282 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:27.585593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476ad94-39aa-4afc-b3c6-1b6d31cc4282 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:27.587967 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5476ad94-39aa-4afc-b3c6-1b6d31cc4282 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:27.594984 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 472/936] 10.222.0.45 () {56 vars in 1126 bytes} [Thu Aug 4 02:05:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:27.601278 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476ad94-39aa-4afc-b3c6-1b6d31cc4282 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:27.607735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:27.690394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a839a-83dd-4df5-8ed0-d2ea5e40a908 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:27.727352 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2644af-0b24-442f-980f-0c0b92025d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:27.727710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2644af-0b24-442f-980f-0c0b92025d5c None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:27.727969 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2644af-0b24-442f-980f-0c0b92025d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:27.728305 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2644af-0b24-442f-980f-0c0b92025d5c None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:27.729140 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2644af-0b24-442f-980f-0c0b92025d5c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:27.729497 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2644af-0b24-442f-980f-0c0b92025d5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:27.730933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2644af-0b24-442f-980f-0c0b92025d5c None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:27.744140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5476ad94-39aa-4afc-b3c6-1b6d31cc4282 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:27.755553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 465/937] 10.222.0.45 () {56 vars in 1149 bytes} [Thu Aug 4 02:05:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:27.756790 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2644af-0b24-442f-980f-0c0b92025d5c None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:27.782514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:27.796153 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e89af-8a09-4e78-a9fe-cea4f3dff087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:27.797150 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e89af-8a09-4e78-a9fe-cea4f3dff087 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:27.797651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e89af-8a09-4e78-a9fe-cea4f3dff087 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:27.798150 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e89af-8a09-4e78-a9fe-cea4f3dff087 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:27.799141 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e89af-8a09-4e78-a9fe-cea4f3dff087 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:27.800221 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e89af-8a09-4e78-a9fe-cea4f3dff087 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:27.802441 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e89af-8a09-4e78-a9fe-cea4f3dff087 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:27.810443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 473/938] 10.222.0.45 () {56 vars in 1126 bytes} [Thu Aug 4 02:05:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:27.817681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e89af-8a09-4e78-a9fe-cea4f3dff087 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:27.862770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:27.874720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2644af-0b24-442f-980f-0c0b92025d5c None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:27.916773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f750b-43e5-4c62-8bd2-63585d65b840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:27.918268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f750b-43e5-4c62-8bd2-63585d65b840 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:27.918999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f750b-43e5-4c62-8bd2-63585d65b840 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:27.919505 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51f750b-43e5-4c62-8bd2-63585d65b840 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:27.935373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f750b-43e5-4c62-8bd2-63585d65b840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:27.936533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e89af-8a09-4e78-a9fe-cea4f3dff087 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:27.937394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51f750b-43e5-4c62-8bd2-63585d65b840 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:27.959465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-d51f750b-43e5-4c62-8bd2-63585d65b840 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:27.959465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:27.959465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:27.959465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:27.959465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:27.959465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:27.959465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:27.959465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:27.959465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:27.959465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:27.959465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:27.959465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:27.959465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:27.959465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:27.959465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:27.960442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:27.960442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:27.960442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:27.960442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:27.960442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:27.960442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:27.960442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:27.960442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:27.960442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:27.960442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:27.960442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:27.960442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:27.960442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:27.960442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:27.960442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:27.960442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:27.960442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:27.961390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:27.961390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:27.961390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:27.961390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:27.962253 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 466/939] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:27.970485 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f750b-43e5-4c62-8bd2-63585d65b840 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:27.978444 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fce10-7cb9-4ba4-926d-f7067175d454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:27.979141 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fce10-7cb9-4ba4-926d-f7067175d454 None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:27.979485 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fce10-7cb9-4ba4-926d-f7067175d454 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:27.979821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fce10-7cb9-4ba4-926d-f7067175d454 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:27.980599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581fce10-7cb9-4ba4-926d-f7067175d454 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:27.982034 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581fce10-7cb9-4ba4-926d-f7067175d454 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:28.016733 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 474/940] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:05:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:05:28.111752 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51f750b-43e5-4c62-8bd2-63585d65b840 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:28.156020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721748ee-cced-48ea-8fc2-8008fc3964f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:28.156935 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721748ee-cced-48ea-8fc2-8008fc3964f6 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:28.157358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721748ee-cced-48ea-8fc2-8008fc3964f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:28.157802 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721748ee-cced-48ea-8fc2-8008fc3964f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:28.158651 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721748ee-cced-48ea-8fc2-8008fc3964f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:28.159706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721748ee-cced-48ea-8fc2-8008fc3964f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:28.161260 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721748ee-cced-48ea-8fc2-8008fc3964f6 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:28.171780 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:28.177095 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 467/941] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:28.184535 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fce10-7cb9-4ba4-926d-f7067175d454 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:28.239761 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:28.306458 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fce10-7cb9-4ba4-926d-f7067175d454 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:28.344387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7633f1-86fa-4f3b-ac3d-8cda45448344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:28.345065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7633f1-86fa-4f3b-ac3d-8cda45448344 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:28.345409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7633f1-86fa-4f3b-ac3d-8cda45448344 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:28.345686 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7633f1-86fa-4f3b-ac3d-8cda45448344 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:28.353506 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7633f1-86fa-4f3b-ac3d-8cda45448344 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:28.354955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7633f1-86fa-4f3b-ac3d-8cda45448344 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:28.367408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-7e7633f1-86fa-4f3b-ac3d-8cda45448344 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:28.367408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:28.367408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:28.367408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:28.367408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:28.367408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:28.367408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:28.367408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:28.367408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:28.367408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:28.367408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:28.367408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:28.367408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:05:28.367408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:05:28.367408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:05:28.368427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:05:28.368427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:28.368427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:28.368427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:28.368427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:28.368427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:28.368427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:28.368427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:28.368427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:28.368427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:28.368427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:28.368427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:28.368427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:28.368427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:28.368427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:28.368427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:28.368427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:28.369409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:05:28.369409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:05:28.369409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:05:28.369409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:05:28.369409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:05:28.369409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:05:28.369409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:28.369409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:28.369409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 475/942] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:28.376115 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721748ee-cced-48ea-8fc2-8008fc3964f6 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:28.485056 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721748ee-cced-48ea-8fc2-8008fc3964f6 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:28.521126 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4c138-122a-4987-afa4-d41d5a85aebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:28.522148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4c138-122a-4987-afa4-d41d5a85aebd None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:28.522626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4c138-122a-4987-afa4-d41d5a85aebd None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:28.523109 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4c138-122a-4987-afa4-d41d5a85aebd None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:28.524049 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4c138-122a-4987-afa4-d41d5a85aebd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:28.524549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4c138-122a-4987-afa4-d41d5a85aebd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:28.526189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4c138-122a-4987-afa4-d41d5a85aebd None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:28.539676 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 468/943] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:05:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:28.545800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7633f1-86fa-4f3b-ac3d-8cda45448344 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:28.655171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7633f1-86fa-4f3b-ac3d-8cda45448344 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:28.666326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:28.701814 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c797ce-9ff2-43e2-bfe5-e2c15977392e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:28.702485 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c797ce-9ff2-43e2-bfe5-e2c15977392e None admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:28.702765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c797ce-9ff2-43e2-bfe5-e2c15977392e None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:28.703041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c797ce-9ff2-43e2-bfe5-e2c15977392e None admin] PATH_INFO: `/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/e54e713b52e84393b018807f8153922e/roles/609fcaecc1a648f8b41c081fe0703ef4` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:28.705307 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c797ce-9ff2-43e2-bfe5-e2c15977392e None admin] RBAC: Authorizing `identity:create_grant(project_id=74b8caff828a4f5289094fb2ae2278f7, user_id=e54e713b52e84393b018807f8153922e, role_id=609fcaecc1a648f8b41c081fe0703ef4)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:28.707326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c797ce-9ff2-43e2-bfe5-e2c15977392e None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:28.722997 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 476/944] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:05:28 2022] PUT /identity/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/e54e713b52e84393b018807f8153922e/roles/609fcaecc1a648f8b41c081fe0703ef4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:05:28.783546 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be3563-5dbd-4cda-bdf6-aa74103668a2 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:28.784107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be3563-5dbd-4cda-bdf6-aa74103668a2 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:28.784107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9be3563-5dbd-4cda-bdf6-aa74103668a2 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:28.785833 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 469/945] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:28.791476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0d26c-d61f-4abf-ba45-225b6759ca7a None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:28.791975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0d26c-d61f-4abf-ba45-225b6759ca7a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:28.792076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0d26c-d61f-4abf-ba45-225b6759ca7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:28.812207 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-49d0d26c-d61f-4abf-ba45-225b6759ca7a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:28.813725 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-49d0d26c-d61f-4abf-ba45-225b6759ca7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:28.907516 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 477/946] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:28.930474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be3563-5dbd-4cda-bdf6-aa74103668a2 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:28.994718 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:29.044366 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9be3563-5dbd-4cda-bdf6-aa74103668a2 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:29.082242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229431ac-9e16-4bf9-ae1e-de0a5941fcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:29.083266 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229431ac-9e16-4bf9-ae1e-de0a5941fcc7 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:29.083759 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229431ac-9e16-4bf9-ae1e-de0a5941fcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:29.084232 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229431ac-9e16-4bf9-ae1e-de0a5941fcc7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:29.085088 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229431ac-9e16-4bf9-ae1e-de0a5941fcc7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:29.086655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229431ac-9e16-4bf9-ae1e-de0a5941fcc7 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:29.095950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-229431ac-9e16-4bf9-ae1e-de0a5941fcc7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 02:05:29.095950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:29.095950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:29.095950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:29.095950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:29.095950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:29.095950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:29.095950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:29.095950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 02:05:29.095950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:29.095950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:29.095950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:29.095950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:29.095950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:29.095950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:29.095950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:29.095950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:29.097571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:29.097571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:29.097571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:29.097571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:29.097571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:29.097571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:29.097571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:29.097571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:29.097571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:29.097571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:29.097571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:29.097571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:29.097571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:29.097571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:29.097571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:29.097571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:29.098480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:29.098480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:29.098480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:29.098480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:29.098480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:29.098480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:29.098480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:29.098480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:29.098480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:29.098480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:29.098480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:29.098480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:29.098480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 02:05:29.098480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:29.101305 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 470/947] 10.222.0.45 () {56 vars in 1126 bytes} [Thu Aug 4 02:05:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:29.113554 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0d26c-d61f-4abf-ba45-225b6759ca7a None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:29.213978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:29.218840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d0d26c-d61f-4abf-ba45-225b6759ca7a None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:29.258125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a4b6bb-0461-4a72-9941-048b407420b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:29.259156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4b6bb-0461-4a72-9941-048b407420b8 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:29.259644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4b6bb-0461-4a72-9941-048b407420b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:29.260125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a4b6bb-0461-4a72-9941-048b407420b8 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:29.261035 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4b6bb-0461-4a72-9941-048b407420b8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:29.261563 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4b6bb-0461-4a72-9941-048b407420b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:29.263136 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a4b6bb-0461-4a72-9941-048b407420b8 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:29.269463 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 478/948] 10.222.0.45 () {56 vars in 1122 bytes} [Thu Aug 4 02:05:29 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:29.275533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229431ac-9e16-4bf9-ae1e-de0a5941fcc7 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:29.384666 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229431ac-9e16-4bf9-ae1e-de0a5941fcc7 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:29.402482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdd9dd-242a-4574-8daa-402c67773fe8 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:29.402790 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdd9dd-242a-4574-8daa-402c67773fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:29.403033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdd9dd-242a-4574-8daa-402c67773fe8 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:29.404701 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 479/949] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:05:29.409562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:29.410422 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f3f77-c9b0-4fbb-92b1-800590fea107 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:29.410843 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f3f77-c9b0-4fbb-92b1-800590fea107 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:29.411009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733f3f77-c9b0-4fbb-92b1-800590fea107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:29.428111 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7de14-3e14-409c-ac44-9c659f4a4420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:29.429184 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7de14-3e14-409c-ac44-9c659f4a4420 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:29.429679 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7de14-3e14-409c-ac44-9c659f4a4420 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:29.430159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7de14-3e14-409c-ac44-9c659f4a4420 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:29.430891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-733f3f77-c9b0-4fbb-92b1-800590fea107 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:29.431268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7de14-3e14-409c-ac44-9c659f4a4420 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:29.431901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7de14-3e14-409c-ac44-9c659f4a4420 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:29.432080 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-733f3f77-c9b0-4fbb-92b1-800590fea107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:29.433667 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7de14-3e14-409c-ac44-9c659f4a4420 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:29.444772 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 471/950] 10.222.0.45 () {56 vars in 1122 bytes} [Thu Aug 4 02:05:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:29.446983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7de14-3e14-409c-ac44-9c659f4a4420 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:29.516397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 480/951] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 02:05:29.539998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f3f77-c9b0-4fbb-92b1-800590fea107 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:29.571850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7de14-3e14-409c-ac44-9c659f4a4420 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:29.611637 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b720f7f-79da-4b0a-924e-3b254cd85f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:29.612445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b720f7f-79da-4b0a-924e-3b254cd85f79 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:29.612582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b720f7f-79da-4b0a-924e-3b254cd85f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:29.612877 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b720f7f-79da-4b0a-924e-3b254cd85f79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:29.613622 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b720f7f-79da-4b0a-924e-3b254cd85f79 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:29.614952 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b720f7f-79da-4b0a-924e-3b254cd85f79 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:29.648653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 472/952] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:05:29 2022] POST /identity/v3/endpoints => generated 349 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:05:29.690558 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:29.702695 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733f3f77-c9b0-4fbb-92b1-800590fea107 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:29.758787 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c73bc-7287-4d3e-8e7a-9210a6166eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:29.759709 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c73bc-7287-4d3e-8e7a-9210a6166eea None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:29.760215 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c73bc-7287-4d3e-8e7a-9210a6166eea None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:29.760733 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c73bc-7287-4d3e-8e7a-9210a6166eea None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:29.761626 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c73bc-7287-4d3e-8e7a-9210a6166eea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:29.763295 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c73bc-7287-4d3e-8e7a-9210a6166eea None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:29.766252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: WARNING keystone.server.flask.common [None req-f59c73bc-7287-4d3e-8e7a-9210a6166eea None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 04 02:05:29.787729 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.sql.core [None req-f59c73bc-7287-4d3e-8e7a-9210a6166eea None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 02:05:29.787729 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 02:05:29.787729 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [parameters: {'id': '591355f1e5e64814a470fdd63d975981', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 02:05:29.787729 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80672) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 02:05:29.796661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-f59c73bc-7287-4d3e-8e7a-9210a6166eea None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 591355f1e5e64814a470fdd63d975981.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 591355f1e5e64814a470fdd63d975981. Aug 04 02:05:29.796661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:29.796661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 02:05:29.796661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 02:05:29.796661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 02:05:29.796661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 02:05:29.796661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 02:05:29.796661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 02:05:29.796661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 02:05:29.796661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application conn.query(q) Aug 04 02:05:29.796661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 02:05:29.796661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 02:05:29.796661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application result.read() Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 04 02:05:29.797972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 04 02:05:29.798916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 02:05:29.798916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application next(self.gen) Aug 04 02:05:29.798916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 02:05:29.798916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application yield resource Aug 04 02:05:29.798916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 02:05:29.798916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application next(self.gen) Aug 04 02:05:29.798916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 02:05:29.798916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 02:05:29.798916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 02:05:29.798916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 02:05:29.798916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 02:05:29.798916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise self.value Aug 04 02:05:29.798916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 02:05:29.798916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 02:05:29.798916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 02:05:29.798916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application session.commit() Aug 04 02:05:29.798916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 02:05:29.799880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 02:05:29.799880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 02:05:29.799880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 02:05:29.799880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 02:05:29.799880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self.session.flush() Aug 04 02:05:29.799880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 02:05:29.799880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 02:05:29.799880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 02:05:29.799880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 02:05:29.799880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 02:05:29.799880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application compat.raise_( Aug 04 02:05:29.799880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 02:05:29.799880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception Aug 04 02:05:29.799880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 02:05:29.799880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 02:05:29.799880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 02:05:29.799880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 02:05:29.800818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 02:05:29.800818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 02:05:29.800818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 02:05:29.800818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 02:05:29.800818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 04 02:05:29.800818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 04 02:05:29.800818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 02:05:29.800818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 02:05:29.800818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 02:05:29.800818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 02:05:29.800818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 02:05:29.800818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 02:05:29.800818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 02:05:29.800818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 02:05:29.800818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 02:05:29.801650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 02:05:29.801650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 02:05:29.801650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception Aug 04 02:05:29.801650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 02:05:29.801650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 02:05:29.801650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 02:05:29.801650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 02:05:29.801650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 02:05:29.801650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 02:05:29.801650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 02:05:29.801650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application conn.query(q) Aug 04 02:05:29.801650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 02:05:29.801650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 02:05:29.801650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 02:05:29.801650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application result.read() Aug 04 02:05:29.801650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 02:05:29.801650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 02:05:29.803598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 02:05:29.803598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 02:05:29.803598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 02:05:29.803598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 02:05:29.803598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 02:05:29.803598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 02:05:29.803598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 04 02:05:29.803598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 04 02:05:29.803598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [parameters: {'id': '591355f1e5e64814a470fdd63d975981', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 04 02:05:29.803598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 02:05:29.803598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:29.803598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:29.803598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:29.803598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:29.803598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 04 02:05:29.805618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 04 02:05:29.805618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 02:05:29.805618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 02:05:29.805618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 04 02:05:29.805618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:29.805618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:29.805618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:29.805618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:29.805618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:29.805618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:29.805618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:29.805618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:29.805618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:29.805618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:29.805618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:29.805618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:29.805618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:29.807861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:29.807861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 04 02:05:29.807861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 04 02:05:29.807861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:29.807861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:29.807861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 04 02:05:29.807861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 04 02:05:29.807861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 591355f1e5e64814a470fdd63d975981. Aug 04 02:05:29.807861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:29.810748 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 481/953] 10.222.0.45 () {60 vars in 1140 bytes} [Thu Aug 4 02:05:29 2022] POST /identity/v3/projects => generated 262 bytes in 271 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:29.817239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b720f7f-79da-4b0a-924e-3b254cd85f79 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:29.869571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:29.935895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b720f7f-79da-4b0a-924e-3b254cd85f79 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:29.973176 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc6efdd-c120-4ce9-ae7c-a1368016b48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:29.974098 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6efdd-c120-4ce9-ae7c-a1368016b48c None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:29.974531 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6efdd-c120-4ce9-ae7c-a1368016b48c None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:29.975019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6efdd-c120-4ce9-ae7c-a1368016b48c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:29.982502 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6efdd-c120-4ce9-ae7c-a1368016b48c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:29.984334 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc6efdd-c120-4ce9-ae7c-a1368016b48c None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:29.996893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-5cc6efdd-c120-4ce9-ae7c-a1368016b48c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:29.996893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:29.996893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:29.996893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:29.996893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:29.996893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:29.996893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:29.996893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:29.996893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:29.996893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:29.996893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:29.996893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:29.996893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:05:29.996893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:05:29.996893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:05:29.998194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:05:29.998194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:29.998194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:29.998194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:29.998194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:29.998194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:29.998194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:29.998194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:29.998194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:29.998194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:29.998194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:29.998194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:29.998194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:29.998194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:29.998194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:29.998194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:29.998194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:29.999384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:05:29.999384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:05:29.999384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:05:29.999384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:05:29.999384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:05:29.999384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:05:29.999384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:29.999384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:30.001585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 473/954] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:30.012759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c73bc-7287-4d3e-8e7a-9210a6166eea None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:30.121299 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c73bc-7287-4d3e-8e7a-9210a6166eea None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:30.156412 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc17eb-6295-4173-bd91-7889f3b2f41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:30.157070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc17eb-6295-4173-bd91-7889f3b2f41f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:30.157348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc17eb-6295-4173-bd91-7889f3b2f41f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:30.157620 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc17eb-6295-4173-bd91-7889f3b2f41f None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:30.158339 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc17eb-6295-4173-bd91-7889f3b2f41f None admin] RBAC: Adding query filter params (name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:30.158643 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc17eb-6295-4173-bd91-7889f3b2f41f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:30.160808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc17eb-6295-4173-bd91-7889f3b2f41f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:30.182070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 482/955] 10.222.0.45 () {56 vars in 1112 bytes} [Thu Aug 4 02:05:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:30.239222 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50240f-f068-442c-95ea-0f5ac5e58d0e None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:30.239580 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50240f-f068-442c-95ea-0f5ac5e58d0e None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:30.239770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50240f-f068-442c-95ea-0f5ac5e58d0e None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:30.241518 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 474/956] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:30.247294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3e364-6b10-4375-a9ab-3797619e0378 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:30.247621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3e364-6b10-4375-a9ab-3797619e0378 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:30.247858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e3e364-6b10-4375-a9ab-3797619e0378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:30.255583 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:30.270728 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-f7e3e364-6b10-4375-a9ab-3797619e0378 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:30.272343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f7e3e364-6b10-4375-a9ab-3797619e0378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:30.356275 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 483/957] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:30.379267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50240f-f068-442c-95ea-0f5ac5e58d0e None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:30.488115 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50240f-f068-442c-95ea-0f5ac5e58d0e None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:30.490608 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:30.525668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec8729-c50a-4029-b34d-6ee1373a5774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:30.526313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec8729-c50a-4029-b34d-6ee1373a5774 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:30.526592 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec8729-c50a-4029-b34d-6ee1373a5774 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:30.526864 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec8729-c50a-4029-b34d-6ee1373a5774 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:30.527493 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec8729-c50a-4029-b34d-6ee1373a5774 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:30.528889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec8729-c50a-4029-b34d-6ee1373a5774 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:30.538127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-82ec8729-c50a-4029-b34d-6ee1373a5774 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 02:05:30.538127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:30.538127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:30.538127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:30.538127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:30.538127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:30.538127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:30.538127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:30.538127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 02:05:30.538127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:30.538127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:30.538127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:30.538127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:30.538127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:30.538127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:30.538127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:30.538127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:30.539237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:30.539237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:30.539237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:30.539237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:30.539237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:30.539237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:30.539237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:30.539237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:30.539237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:30.539237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:30.539237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:30.539237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:30.539237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:30.539237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:30.539237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:30.539237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:30.540097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:30.540097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:30.540097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:30.540097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:30.540097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:30.540097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:30.540097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:30.540097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:30.540097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:30.540097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:30.540097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:30.540097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:30.540097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 02:05:30.540097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:30.540097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 475/958] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:30.545723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3e364-6b10-4375-a9ab-3797619e0378 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:30.655576 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e3e364-6b10-4375-a9ab-3797619e0378 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:30.689413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e451a-18f2-449b-9fdb-0af3ad019cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:30.690074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e451a-18f2-449b-9fdb-0af3ad019cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:30.690356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e451a-18f2-449b-9fdb-0af3ad019cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:30.690628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e451a-18f2-449b-9fdb-0af3ad019cd5 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:30.691305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e451a-18f2-449b-9fdb-0af3ad019cd5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:30.691606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e451a-18f2-449b-9fdb-0af3ad019cd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:30.692984 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e451a-18f2-449b-9fdb-0af3ad019cd5 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:30.694264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:30.702865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 484/959] 10.222.0.45 () {56 vars in 1112 bytes} [Thu Aug 4 02:05:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:30.704202 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec8729-c50a-4029-b34d-6ee1373a5774 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:30.810204 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec8729-c50a-4029-b34d-6ee1373a5774 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:30.845657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f0a17-376d-48e9-bf38-7008ef0ce8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:30.847608 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f0a17-376d-48e9-bf38-7008ef0ce8fe None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:30.848103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f0a17-376d-48e9-bf38-7008ef0ce8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:30.848584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f0a17-376d-48e9-bf38-7008ef0ce8fe None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:30.849537 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f0a17-376d-48e9-bf38-7008ef0ce8fe None admin] RBAC: Adding query filter params (type=compute) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:30.850058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f0a17-376d-48e9-bf38-7008ef0ce8fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:30.851634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f0a17-376d-48e9-bf38-7008ef0ce8fe None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:30.857066 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 476/960] 10.222.0.45 () {56 vars in 1112 bytes} [Thu Aug 4 02:05:30 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:30.859751 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dd0eb-1b84-4d12-ad2b-bf21460abaa5 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:30.860234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dd0eb-1b84-4d12-ad2b-bf21460abaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:30.860322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813dd0eb-1b84-4d12-ad2b-bf21460abaa5 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:30.866896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 485/961] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:30.868894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf840743-c4f6-4bf8-8a5f-e6f424ee7062 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:30.869491 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf840743-c4f6-4bf8-8a5f-e6f424ee7062 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:30.869992 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf840743-c4f6-4bf8-8a5f-e6f424ee7062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:30.889836 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-cf840743-c4f6-4bf8-8a5f-e6f424ee7062 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:30.891502 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-cf840743-c4f6-4bf8-8a5f-e6f424ee7062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:30.929039 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:30.972474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 477/962] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:30.995119 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dd0eb-1b84-4d12-ad2b-bf21460abaa5 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:31.108307 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813dd0eb-1b84-4d12-ad2b-bf21460abaa5 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:31.142999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9c9d6-513c-40f9-a91c-4fbfba955d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:31.144153 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9c9d6-513c-40f9-a91c-4fbfba955d8a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:31.144433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9c9d6-513c-40f9-a91c-4fbfba955d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:31.144757 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9c9d6-513c-40f9-a91c-4fbfba955d8a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:31.155185 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e9c9d6-513c-40f9-a91c-4fbfba955d8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:05:31.160745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-76e9c9d6-513c-40f9-a91c-4fbfba955d8a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 02:05:31.160745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:31.160745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:31.160745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:31.160745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:31.160745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:31.160745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:31.160745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:31.160745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:31.160745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:31.160745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:31.160745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:31.160745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:05:31.160745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:05:31.160745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:05:31.160745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:05:31.161721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:05:31.161721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:05:31.161721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:05:31.161721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:05:31.161721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:31.161721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:31.161721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:31.161721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:31.161721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:31.161721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:31.161721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:31.161721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:31.161721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:31.161721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:31.161721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:31.161721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:31.162595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:31.162595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:31.162595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:31.162595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:31.162595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:05:31.162595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:05:31.162595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:05:31.162595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:05:31.162595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:05:31.162595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:05:31.162595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 02:05:31.162595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:31.162595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 486/963] 10.222.0.45 () {56 vars in 1120 bytes} [Thu Aug 4 02:05:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:31.168141 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf840743-c4f6-4bf8-8a5f-e6f424ee7062 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:31.269916 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf840743-c4f6-4bf8-8a5f-e6f424ee7062 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:31.305002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff37a5-dd07-4843-bdef-19bc6ba3b5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:31.305625 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff37a5-dd07-4843-bdef-19bc6ba3b5cd None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:31.305898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff37a5-dd07-4843-bdef-19bc6ba3b5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:31.306165 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dff37a5-dd07-4843-bdef-19bc6ba3b5cd None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:31.306882 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff37a5-dd07-4843-bdef-19bc6ba3b5cd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:31.307176 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff37a5-dd07-4843-bdef-19bc6ba3b5cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:31.308546 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dff37a5-dd07-4843-bdef-19bc6ba3b5cd None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:31.318465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 478/964] 10.222.0.45 () {56 vars in 1115 bytes} [Thu Aug 4 02:05:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:31.330010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9c9d6-513c-40f9-a91c-4fbfba955d8a None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:31.397613 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:31.452775 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9c9d6-513c-40f9-a91c-4fbfba955d8a None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:31.491840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e38d4-6f59-40c1-a1ec-6da1649a575f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:31.492482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e38d4-6f59-40c1-a1ec-6da1649a575f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:31.492787 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e38d4-6f59-40c1-a1ec-6da1649a575f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:31.493054 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e38d4-6f59-40c1-a1ec-6da1649a575f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:31.504640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e38d4-6f59-40c1-a1ec-6da1649a575f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:31.506079 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e38d4-6f59-40c1-a1ec-6da1649a575f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:31.522129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-b49e38d4-6f59-40c1-a1ec-6da1649a575f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:31.522129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:31.522129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:31.522129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:31.522129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:31.522129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:31.522129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:31.522129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:31.522129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:31.522129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:31.522129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:31.522129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:31.522129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:31.522129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:31.522129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:31.524288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:31.524288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:31.524288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:31.524288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:31.524288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:31.524288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:31.524288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:31.524288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:31.524288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:31.524288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:31.524288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:31.524288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:31.524288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:31.524288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:31.524288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:31.524288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:31.524288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:31.525352 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:31.525352 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:31.525352 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:31.525352 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:31.525352 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 487/965] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:31.528880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff37a5-dd07-4843-bdef-19bc6ba3b5cd None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:31.577041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:31.630151 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dff37a5-dd07-4843-bdef-19bc6ba3b5cd None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:31.668298 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038becd8-1384-4feb-9800-9020d942e3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:31.668945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038becd8-1384-4feb-9800-9020d942e3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:31.669225 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038becd8-1384-4feb-9800-9020d942e3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:31.669491 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038becd8-1384-4feb-9800-9020d942e3c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:31.670176 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038becd8-1384-4feb-9800-9020d942e3c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:31.670476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038becd8-1384-4feb-9800-9020d942e3c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:31.671837 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038becd8-1384-4feb-9800-9020d942e3c1 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:31.684147 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 479/966] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:31.689933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e38d4-6f59-40c1-a1ec-6da1649a575f None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:31.805457 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e38d4-6f59-40c1-a1ec-6da1649a575f None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:31.817089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:31.842295 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15017da7-70a5-45eb-815e-b36332ccd5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:31.842929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15017da7-70a5-45eb-815e-b36332ccd5aa None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:31.843202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15017da7-70a5-45eb-815e-b36332ccd5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:31.843469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15017da7-70a5-45eb-815e-b36332ccd5aa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:31.851965 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15017da7-70a5-45eb-815e-b36332ccd5aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:31.853413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15017da7-70a5-45eb-815e-b36332ccd5aa None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:31.864699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-15017da7-70a5-45eb-815e-b36332ccd5aa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 02:05:31.864699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:31.864699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:31.864699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:31.864699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:31.864699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:31.864699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:31.864699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:31.864699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:31.864699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:31.864699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:31.864699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:31.864699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:05:31.864699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:05:31.864699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:05:31.864699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:31.867287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:31.869581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:31.869581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:31.869581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:05:31.869581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:05:31.869581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:05:31.869581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:05:31.869581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:05:31.869581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:05:31.869581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 04 02:05:31.869581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:31.869581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 488/967] 10.222.0.45 () {56 vars in 1117 bytes} [Thu Aug 4 02:05:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:31.873762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038becd8-1384-4feb-9800-9020d942e3c1 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:31.942691 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51687cc5-4c2f-473a-8e1d-d7d561dff361 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:31.942691 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51687cc5-4c2f-473a-8e1d-d7d561dff361 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:31.942691 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51687cc5-4c2f-473a-8e1d-d7d561dff361 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:31.943585 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 489/968] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:05:31.948923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951642e-6ef8-43fc-bbf4-0f3c67766005 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:31.949451 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951642e-6ef8-43fc-bbf4-0f3c67766005 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:31.949949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951642e-6ef8-43fc-bbf4-0f3c67766005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:31.969579 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:31.972171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-d951642e-6ef8-43fc-bbf4-0f3c67766005 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:31.974028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d951642e-6ef8-43fc-bbf4-0f3c67766005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:31.989296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038becd8-1384-4feb-9800-9020d942e3c1 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:32.033548 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9461f-e228-4417-b056-d2c2eeb6cd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:32.034707 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9461f-e228-4417-b056-d2c2eeb6cd7c None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:32.034981 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9461f-e228-4417-b056-d2c2eeb6cd7c None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:32.035258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9461f-e228-4417-b056-d2c2eeb6cd7c None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:32.036575 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9461f-e228-4417-b056-d2c2eeb6cd7c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:32.036905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9461f-e228-4417-b056-d2c2eeb6cd7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:32.038615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9461f-e228-4417-b056-d2c2eeb6cd7c None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:32.057108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 490/969] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 02:05:32.059374 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 480/970] 10.222.0.45 () {56 vars in 1149 bytes} [Thu Aug 4 02:05:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:32.064940 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951642e-6ef8-43fc-bbf4-0f3c67766005 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:32.080354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9461f-e228-4417-b056-d2c2eeb6cd7c None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:32.202034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9461f-e228-4417-b056-d2c2eeb6cd7c None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:32.202930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:32.208471 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951642e-6ef8-43fc-bbf4-0f3c67766005 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:32.241247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef5f57-1f0f-4426-844e-1e420feab0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:32.242054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef5f57-1f0f-4426-844e-1e420feab0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:32.242431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef5f57-1f0f-4426-844e-1e420feab0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:32.242830 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef5f57-1f0f-4426-844e-1e420feab0e5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:32.243608 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef5f57-1f0f-4426-844e-1e420feab0e5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:32.245228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef5f57-1f0f-4426-844e-1e420feab0e5 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:32.253546 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68afaf-ff96-4f8c-a72a-24d6d861c9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:32.254383 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68afaf-ff96-4f8c-a72a-24d6d861c9da None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:32.254829 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68afaf-ff96-4f8c-a72a-24d6d861c9da None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:32.255251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68afaf-ff96-4f8c-a72a-24d6d861c9da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:32.256340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-69ef5f57-1f0f-4426-844e-1e420feab0e5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 02:05:32.256340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:32.256340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:32.256340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:32.256340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:32.256340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:32.256340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:32.256340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:32.256340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 02:05:32.256340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:32.256340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:32.256340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:32.256340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:32.256340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:32.256340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:32.256340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:32.256340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:32.257562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:32.257562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:32.257562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:32.257562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:32.257562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:32.257562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:32.257562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:32.257562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:32.257562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:32.257562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:32.257562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:32.257562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:32.257562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:32.257562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:32.257562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:32.257562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:32.258626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:32.258626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:32.258626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:32.258626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:32.258626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:32.258626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:32.258626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:32.258626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:32.258626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:32.258626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:32.258626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:32.258626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:32.258626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 02:05:32.258626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:32.265616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 481/971] 10.222.0.45 () {56 vars in 1126 bytes} [Thu Aug 4 02:05:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:32.266958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef5f57-1f0f-4426-844e-1e420feab0e5 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:32.282412 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:32.343585 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed68afaf-ff96-4f8c-a72a-24d6d861c9da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:32.347454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed68afaf-ff96-4f8c-a72a-24d6d861c9da None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:32.416091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef5f57-1f0f-4426-844e-1e420feab0e5 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:32.464781 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885fcfab-f81f-4708-9117-d517c215e385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:32.465814 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fcfab-f81f-4708-9117-d517c215e385 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:32.466306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fcfab-f81f-4708-9117-d517c215e385 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:32.466929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885fcfab-f81f-4708-9117-d517c215e385 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:32.467845 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885fcfab-f81f-4708-9117-d517c215e385 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:32.468337 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885fcfab-f81f-4708-9117-d517c215e385 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:32.470162 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885fcfab-f81f-4708-9117-d517c215e385 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:32.475634 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:32.478277 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 482/972] 10.222.0.45 () {56 vars in 1122 bytes} [Thu Aug 4 02:05:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:32.479874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc5757-69a4-4337-97c7-5d0597db6493 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:32.480331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc5757-69a4-4337-97c7-5d0597db6493 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:32.480801 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccc5757-69a4-4337-97c7-5d0597db6493 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:32.486272 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 483/973] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:32.488265 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 491/974] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:05:32 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:05:32.489006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc5757-69a4-4337-97c7-5d0597db6493 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:32.490304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71f770-f787-4e51-a434-34ba1a383474 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:32.490737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71f770-f787-4e51-a434-34ba1a383474 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:32.491146 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f71f770-f787-4e51-a434-34ba1a383474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:32.514310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-0f71f770-f787-4e51-a434-34ba1a383474 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:32.515918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-0f71f770-f787-4e51-a434-34ba1a383474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:32.571724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:32.592243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 492/975] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:32.592243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71f770-f787-4e51-a434-34ba1a383474 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:32.619157 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccc5757-69a4-4337-97c7-5d0597db6493 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:32.660147 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725c2f8-97bd-4e38-9720-712bfd4136d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:32.660807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725c2f8-97bd-4e38-9720-712bfd4136d4 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:32.661096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725c2f8-97bd-4e38-9720-712bfd4136d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:32.661368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725c2f8-97bd-4e38-9720-712bfd4136d4 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:32.662084 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725c2f8-97bd-4e38-9720-712bfd4136d4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:32.662354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725c2f8-97bd-4e38-9720-712bfd4136d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:32.663733 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725c2f8-97bd-4e38-9720-712bfd4136d4 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:32.669657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 484/976] 10.222.0.45 () {56 vars in 1122 bytes} [Thu Aug 4 02:05:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:32.673937 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725c2f8-97bd-4e38-9720-712bfd4136d4 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:32.698199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f71f770-f787-4e51-a434-34ba1a383474 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:32.730175 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:32.738303 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeddc96-657b-4a72-a2b4-732268ac07d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:32.738946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeddc96-657b-4a72-a2b4-732268ac07d3 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:32.739226 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeddc96-657b-4a72-a2b4-732268ac07d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:32.739498 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeddc96-657b-4a72-a2b4-732268ac07d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:32.750208 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeddc96-657b-4a72-a2b4-732268ac07d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:32.751666 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeddc96-657b-4a72-a2b4-732268ac07d3 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:32.767012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-bfeddc96-657b-4a72-a2b4-732268ac07d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:32.767012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:32.767012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:32.767012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:32.767012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:32.767012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:32.767012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:32.767012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:32.767012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:32.767012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:32.767012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:32.767012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:32.767012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:32.767012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:32.767012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:32.767968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:32.767968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:32.767968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:32.767968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:32.767968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:32.767968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:32.767968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:32.767968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:32.767968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:32.767968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:32.767968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:32.767968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:32.767968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:32.767968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:32.767968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:32.767968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:32.767968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:32.769582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:32.769582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:32.769582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:32.769582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:32.769582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 493/977] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:32.770534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeddc96-657b-4a72-a2b4-732268ac07d3 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:32.788836 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725c2f8-97bd-4e38-9720-712bfd4136d4 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:32.831732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049dc8f8-1153-4137-988e-e27873c03d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:32.832358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dc8f8-1153-4137-988e-e27873c03d3a None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:32.833486 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dc8f8-1153-4137-988e-e27873c03d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:32.833787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049dc8f8-1153-4137-988e-e27873c03d3a None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:32.834447 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049dc8f8-1153-4137-988e-e27873c03d3a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:32.837033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049dc8f8-1153-4137-988e-e27873c03d3a None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:32.840840 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: WARNING keystone.server.flask.common [None req-049dc8f8-1153-4137-988e-e27873c03d3a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 04 02:05:32.849390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:32.856353 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.sql.core [None req-049dc8f8-1153-4137-988e-e27873c03d3a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 02:05:32.856353 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 02:05:32.856353 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [parameters: {'user_id': 'e8e9f0579328414281d5c5976258f884', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 02:05:32.856353 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80673) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 04 02:05:32.874979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-049dc8f8-1153-4137-988e-e27873c03d3a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 02:05:32.874979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:32.874979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 02:05:32.874979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 02:05:32.874979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 02:05:32.874979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 02:05:32.874979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 02:05:32.874979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 02:05:32.874979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 02:05:32.874979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application conn.query(q) Aug 04 02:05:32.874979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 02:05:32.874979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 02:05:32.874979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 02:05:32.874979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application result.read() Aug 04 02:05:32.874979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 02:05:32.876430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application next(self.gen) Aug 04 02:05:32.877453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 04 02:05:32.877453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application yield resource Aug 04 02:05:32.877453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 04 02:05:32.877453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application next(self.gen) Aug 04 02:05:32.877453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 04 02:05:32.877453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self.session.rollback() Aug 04 02:05:32.877453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 04 02:05:32.877453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self.force_reraise() Aug 04 02:05:32.877453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 04 02:05:32.877453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise self.value Aug 04 02:05:32.877453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 04 02:05:32.877453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 04 02:05:32.877453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 04 02:05:32.877453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application session.commit() Aug 04 02:05:32.877453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 04 02:05:32.877453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 04 02:05:32.877453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 04 02:05:32.879345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._prepare_impl() Aug 04 02:05:32.879345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 04 02:05:32.879345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self.session.flush() Aug 04 02:05:32.879345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 04 02:05:32.879345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._flush(objects) Aug 04 02:05:32.879345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 04 02:05:32.879345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 04 02:05:32.879345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 04 02:05:32.879345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application compat.raise_( Aug 04 02:05:32.879345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 02:05:32.879345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception Aug 04 02:05:32.879345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 04 02:05:32.879345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application flush_context.execute() Aug 04 02:05:32.879345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 04 02:05:32.879345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rec.execute(self) Aug 04 02:05:32.879345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 04 02:05:32.879345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 04 02:05:32.880508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 04 02:05:32.880508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 04 02:05:32.880508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 04 02:05:32.880508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 04 02:05:32.880508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 04 02:05:32.880508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 04 02:05:32.880508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 04 02:05:32.880508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 04 02:05:32.880508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 04 02:05:32.880508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 04 02:05:32.880508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 04 02:05:32.880508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 04 02:05:32.880508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 04 02:05:32.880508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 04 02:05:32.880508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 04 02:05:32.880508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception Aug 04 02:05:32.881685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 04 02:05:32.881685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 04 02:05:32.881685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 04 02:05:32.881685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 04 02:05:32.881685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 04 02:05:32.881685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application result = self._query(query) Aug 04 02:05:32.881685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 04 02:05:32.881685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application conn.query(q) Aug 04 02:05:32.881685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 04 02:05:32.881685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 04 02:05:32.881685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 04 02:05:32.881685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application result.read() Aug 04 02:05:32.881685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 04 02:05:32.881685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 04 02:05:32.881685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 04 02:05:32.881685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 04 02:05:32.882724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 04 02:05:32.882724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 04 02:05:32.882724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 04 02:05:32.882724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 04 02:05:32.882724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 04 02:05:32.882724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 04 02:05:32.882724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e8e9f0579328414281d5c5976258f884', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 04 02:05:32.882724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 04 02:05:32.882724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:32.882724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:32.882724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:32.882724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:32.882724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:32.882724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:32.882724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:32.884636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:32.884636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:32.884636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:32.884636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:32.884636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:32.884636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:32.884636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:32.884636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 04 02:05:32.884636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 04 02:05:32.884636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:32.884636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:32.884636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:05:32.884636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:32.884636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:05:32.884636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:32.884636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 04 02:05:32.884636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 04 02:05:32.886913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 04 02:05:32.886913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 04 02:05:32.886913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 04 02:05:32.886913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 04 02:05:32.886913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 04 02:05:32.886913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:32.886913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 485/978] 10.222.0.45 () {60 vars in 1131 bytes} [Thu Aug 4 02:05:32 2022] POST /identity/v3/users => generated 160 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:32.886913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049dc8f8-1153-4137-988e-e27873c03d3a None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:32.895218 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeddc96-657b-4a72-a2b4-732268ac07d3 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:32.934114 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02d646-4ce8-4aa9-9396-18f97967c462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:32.934758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02d646-4ce8-4aa9-9396-18f97967c462 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:32.935041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02d646-4ce8-4aa9-9396-18f97967c462 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:32.935314 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02d646-4ce8-4aa9-9396-18f97967c462 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:32.936197 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02d646-4ce8-4aa9-9396-18f97967c462 None admin] RBAC: Adding query filter params (interface=admin, service_id=184a26d33f4c4d00851117a1c90761b9, region_id=RegionOne) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:32.936658 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02d646-4ce8-4aa9-9396-18f97967c462 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:32.938320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02d646-4ce8-4aa9-9396-18f97967c462 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:32.948730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 494/979] 10.222.0.45 () {56 vars in 1249 bytes} [Thu Aug 4 02:05:32 2022] GET /identity/v3/endpoints?service_id=184a26d33f4c4d00851117a1c90761b9&interface=admin®ion_id=RegionOne => generated 193 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:32.948730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02d646-4ce8-4aa9-9396-18f97967c462 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:32.993690 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049dc8f8-1153-4137-988e-e27873c03d3a None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:32.999301 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:33.029791 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85648e35-035e-494f-80a6-ca8c20fb9d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:33.030443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85648e35-035e-494f-80a6-ca8c20fb9d00 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:33.030720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85648e35-035e-494f-80a6-ca8c20fb9d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:33.030991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85648e35-035e-494f-80a6-ca8c20fb9d00 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:33.031689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85648e35-035e-494f-80a6-ca8c20fb9d00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:33.031991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85648e35-035e-494f-80a6-ca8c20fb9d00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:33.033601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85648e35-035e-494f-80a6-ca8c20fb9d00 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:33.054876 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 486/980] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:33.054876 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85648e35-035e-494f-80a6-ca8c20fb9d00 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:33.064256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02d646-4ce8-4aa9-9396-18f97967c462 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:33.098791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3775a86-4ac2-4fe7-8f40-c76d1ef8f7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:33.099428 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3775a86-4ac2-4fe7-8f40-c76d1ef8f7da None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:33.099707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3775a86-4ac2-4fe7-8f40-c76d1ef8f7da None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:33.099979 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3775a86-4ac2-4fe7-8f40-c76d1ef8f7da None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:33.107654 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3775a86-4ac2-4fe7-8f40-c76d1ef8f7da None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:33.109815 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3775a86-4ac2-4fe7-8f40-c76d1ef8f7da None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:33.115722 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:33.128536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-b3775a86-4ac2-4fe7-8f40-c76d1ef8f7da None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 02:05:33.128536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:33.128536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:33.128536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:33.128536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:33.128536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:33.128536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:33.128536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:33.128536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:33.128536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:33.128536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:33.128536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:33.128536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:05:33.128536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:05:33.128536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:05:33.128536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:33.130509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:33.132025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:33.132025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:33.132025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:05:33.132025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:05:33.132025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:05:33.132025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:05:33.132025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:05:33.132025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:05:33.132025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 04 02:05:33.132025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:33.132025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 495/981] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:05:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:33.132025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3775a86-4ac2-4fe7-8f40-c76d1ef8f7da None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:33.166768 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85648e35-035e-494f-80a6-ca8c20fb9d00 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:33.201148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91f4e0-2bc6-4e29-8599-868f0fe86cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:33.201765 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91f4e0-2bc6-4e29-8599-868f0fe86cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:33.202044 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91f4e0-2bc6-4e29-8599-868f0fe86cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:33.202313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91f4e0-2bc6-4e29-8599-868f0fe86cd8 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:33.202959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91f4e0-2bc6-4e29-8599-868f0fe86cd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:33.204323 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91f4e0-2bc6-4e29-8599-868f0fe86cd8 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:33.211165 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 487/982] 10.222.0.45 () {56 vars in 1087 bytes} [Thu Aug 4 02:05:33 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:05:33.212237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249fe75-6bea-4fd9-b84c-5357aa73db1b None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:33.212519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249fe75-6bea-4fd9-b84c-5357aa73db1b None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:33.212813 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249fe75-6bea-4fd9-b84c-5357aa73db1b None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:33.214045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 488/983] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:33.214765 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249fe75-6bea-4fd9-b84c-5357aa73db1b None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:33.251464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3775a86-4ac2-4fe7-8f40-c76d1ef8f7da None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:33.259545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:33.295140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0e749-582f-47b1-83a2-35cc8806b635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:33.295805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0e749-582f-47b1-83a2-35cc8806b635 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:33.296083 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0e749-582f-47b1-83a2-35cc8806b635 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:33.296546 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0e749-582f-47b1-83a2-35cc8806b635 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:33.304731 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0e749-582f-47b1-83a2-35cc8806b635 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:33.306237 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0e749-582f-47b1-83a2-35cc8806b635 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:33.319252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-45b0e749-582f-47b1-83a2-35cc8806b635 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:33.319252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:33.319252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:33.319252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:33.319252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:33.319252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:33.319252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:33.319252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:33.319252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:33.319252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:33.319252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:33.319252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:33.319252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:05:33.319252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:05:33.319252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:05:33.320452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:05:33.320452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:33.320452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:33.320452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:33.320452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:33.320452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:33.320452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:33.320452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:33.320452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:33.320452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:33.320452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:33.320452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:33.320452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:33.320452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:33.320452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:33.320452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:33.320452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:33.321620 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:05:33.321620 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:05:33.321620 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:05:33.321620 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:05:33.321620 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:05:33.321620 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:05:33.321620 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:33.321620 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:33.325660 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 496/984] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:33.326958 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff2e5a-dbbf-464d-a846-05eaed579c5a None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:33.327250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff2e5a-dbbf-464d-a846-05eaed579c5a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:33.327531 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff2e5a-dbbf-464d-a846-05eaed579c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:33.342905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249fe75-6bea-4fd9-b84c-5357aa73db1b None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:33.348191 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-b4ff2e5a-dbbf-464d-a846-05eaed579c5a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:33.349555 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b4ff2e5a-dbbf-464d-a846-05eaed579c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:33.366486 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:33.386707 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebf151-4c7a-4bb0-bbcd-16ee01c4b636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:33.387374 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebf151-4c7a-4bb0-bbcd-16ee01c4b636 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:33.387660 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebf151-4c7a-4bb0-bbcd-16ee01c4b636 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:33.387934 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aebf151-4c7a-4bb0-bbcd-16ee01c4b636 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:33.388724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebf151-4c7a-4bb0-bbcd-16ee01c4b636 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:33.389034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebf151-4c7a-4bb0-bbcd-16ee01c4b636 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:33.390463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aebf151-4c7a-4bb0-bbcd-16ee01c4b636 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:33.429678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 489/985] 10.222.0.45 () {56 vars in 1101 bytes} [Thu Aug 4 02:05:33 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:33.430528 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebf151-4c7a-4bb0-bbcd-16ee01c4b636 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:33.447466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 497/986] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:33.470826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff2e5a-dbbf-464d-a846-05eaed579c5a None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:33.557666 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aebf151-4c7a-4bb0-bbcd-16ee01c4b636 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:33.588690 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff2e5a-dbbf-464d-a846-05eaed579c5a None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:33.591947 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:33.602618 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18e472-836b-4dd8-ab9b-e30fda8b8243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:33.603680 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18e472-836b-4dd8-ab9b-e30fda8b8243 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:33.603680 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18e472-836b-4dd8-ab9b-e30fda8b8243 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:33.603872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18e472-836b-4dd8-ab9b-e30fda8b8243 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:33.604522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18e472-836b-4dd8-ab9b-e30fda8b8243 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:33.604848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18e472-836b-4dd8-ab9b-e30fda8b8243 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:33.606283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18e472-836b-4dd8-ab9b-e30fda8b8243 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:33.619784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 490/987] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:05:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:33.628154 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18e472-836b-4dd8-ab9b-e30fda8b8243 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:33.642131 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4756cbc-6e6b-4d37-a126-2a2217a5cb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:33.643824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4756cbc-6e6b-4d37-a126-2a2217a5cb0b None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:33.644258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4756cbc-6e6b-4d37-a126-2a2217a5cb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:33.644704 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4756cbc-6e6b-4d37-a126-2a2217a5cb0b None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:33.645750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4756cbc-6e6b-4d37-a126-2a2217a5cb0b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:33.648039 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4756cbc-6e6b-4d37-a126-2a2217a5cb0b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:33.668460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 498/988] 10.222.0.45 () {60 vars in 1141 bytes} [Thu Aug 4 02:05:33 2022] POST /identity/v3/services => generated 242 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:05:33.685846 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:33.766258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18e472-836b-4dd8-ab9b-e30fda8b8243 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:33.811930 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c4ce84-a1b6-4247-bf70-daeaa38fde33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:33.812550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4ce84-a1b6-4247-bf70-daeaa38fde33 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:33.812860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4ce84-a1b6-4247-bf70-daeaa38fde33 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:33.813392 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c4ce84-a1b6-4247-bf70-daeaa38fde33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:33.814197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4ce84-a1b6-4247-bf70-daeaa38fde33 None admin] RBAC: Adding query filter params (role.id=609fcaecc1a648f8b41c081fe0703ef4, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=e54e713b52e84393b018807f8153922e) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:33.814510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4ce84-a1b6-4247-bf70-daeaa38fde33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:33.816542 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c4ce84-a1b6-4247-bf70-daeaa38fde33 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:33.832652 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 491/989] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:05:33 2022] GET /identity/v3/role_assignments?user.id=e54e713b52e84393b018807f8153922e&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=609fcaecc1a648f8b41c081fe0703ef4 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:35.294760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd67850-8249-40c2-ab4e-c8c3b71572e5 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:35.295482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd67850-8249-40c2-ab4e-c8c3b71572e5 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:35.295482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd67850-8249-40c2-ab4e-c8c3b71572e5 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:35.297722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 499/990] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:35.302903 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1752e-e48b-4b28-830c-df44e42b8e6c None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:35.303364 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1752e-e48b-4b28-830c-df44e42b8e6c None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:35.303443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad1752e-e48b-4b28-830c-df44e42b8e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:35.322140 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-dad1752e-e48b-4b28-830c-df44e42b8e6c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:35.323618 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-dad1752e-e48b-4b28-830c-df44e42b8e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:35.395438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 492/991] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:35.417847 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd67850-8249-40c2-ab4e-c8c3b71572e5 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:35.526060 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd67850-8249-40c2-ab4e-c8c3b71572e5 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:35.559746 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a475f3-dde3-4422-a987-7f33a8a382e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:35.560300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a475f3-dde3-4422-a987-7f33a8a382e2 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:35.560576 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a475f3-dde3-4422-a987-7f33a8a382e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:35.560853 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a475f3-dde3-4422-a987-7f33a8a382e2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:35.561478 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a475f3-dde3-4422-a987-7f33a8a382e2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:35.562906 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a475f3-dde3-4422-a987-7f33a8a382e2 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:35.572852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-d7a475f3-dde3-4422-a987-7f33a8a382e2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 02:05:35.572852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:35.572852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:35.572852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:35.572852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:35.572852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:35.572852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:35.572852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:35.572852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 02:05:35.572852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:35.572852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:35.572852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:35.572852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:35.572852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:35.572852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:35.572852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:35.572852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:35.573896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:35.573896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:35.573896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:35.573896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:35.573896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:35.573896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:35.573896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:35.573896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:35.573896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:35.573896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:35.573896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:35.573896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:35.573896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:35.573896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:35.573896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:35.573896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:35.574738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:35.574738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:35.574738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:35.574738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:35.574738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:35.574738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:35.574738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:35.574738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:35.574738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:35.574738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:35.574738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:35.574738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:35.574738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 04 02:05:35.574738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:35.576943 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 500/992] 10.222.0.45 () {56 vars in 1126 bytes} [Thu Aug 4 02:05:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:35.580128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1752e-e48b-4b28-830c-df44e42b8e6c None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:35.607130 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:35.688554 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad1752e-e48b-4b28-830c-df44e42b8e6c None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:35.724934 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05b539-9109-4ae7-b286-e29df29658c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:35.725783 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05b539-9109-4ae7-b286-e29df29658c2 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:35.725900 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05b539-9109-4ae7-b286-e29df29658c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:35.726102 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05b539-9109-4ae7-b286-e29df29658c2 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:35.726784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05b539-9109-4ae7-b286-e29df29658c2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:35.727093 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05b539-9109-4ae7-b286-e29df29658c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:35.727262 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d81579-d4ed-4e27-822e-45fef46d5796 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:35.728101 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d81579-d4ed-4e27-822e-45fef46d5796 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:35.728389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d81579-d4ed-4e27-822e-45fef46d5796 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:35.728480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05b539-9109-4ae7-b286-e29df29658c2 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:35.729941 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 501/993] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:05:35.738200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 493/994] 10.222.0.45 () {56 vars in 1122 bytes} [Thu Aug 4 02:05:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:35.738386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5323d-38fd-4921-a2cd-13ee82ad2ad4 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:35.738386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5323d-38fd-4921-a2cd-13ee82ad2ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:35.738386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5323d-38fd-4921-a2cd-13ee82ad2ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:35.739863 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05b539-9109-4ae7-b286-e29df29658c2 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:35.754074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-1db5323d-38fd-4921-a2cd-13ee82ad2ad4 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:35.755267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1db5323d-38fd-4921-a2cd-13ee82ad2ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:35.770422 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:35.833736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 502/995] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 02:05:35.837044 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:35.846716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05b539-9109-4ae7-b286-e29df29658c2 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:35.855889 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5323d-38fd-4921-a2cd-13ee82ad2ad4 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:35.882728 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b42de-7bb2-495c-989c-4fdc4b62c6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:35.883349 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b42de-7bb2-495c-989c-4fdc4b62c6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:35.883630 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b42de-7bb2-495c-989c-4fdc4b62c6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:35.883906 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b42de-7bb2-495c-989c-4fdc4b62c6d8 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:35.884589 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b42de-7bb2-495c-989c-4fdc4b62c6d8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:35.884919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b42de-7bb2-495c-989c-4fdc4b62c6d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:35.886294 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b42de-7bb2-495c-989c-4fdc4b62c6d8 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:35.896331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 494/996] 10.222.0.45 () {56 vars in 1122 bytes} [Thu Aug 4 02:05:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:35.896331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3005ed6-f67f-48cb-95af-50b16162e13b None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:35.896331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3005ed6-f67f-48cb-95af-50b16162e13b None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:35.896331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3005ed6-f67f-48cb-95af-50b16162e13b None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:35.896331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 495/997] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:35.898027 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3005ed6-f67f-48cb-95af-50b16162e13b None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:35.958826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5323d-38fd-4921-a2cd-13ee82ad2ad4 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:35.992508 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bdd0df-eb4c-4b9b-8245-c49200a02054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:35.993165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdd0df-eb4c-4b9b-8245-c49200a02054 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:35.993444 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdd0df-eb4c-4b9b-8245-c49200a02054 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:35.993714 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdd0df-eb4c-4b9b-8245-c49200a02054 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:35.994334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bdd0df-eb4c-4b9b-8245-c49200a02054 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:35.995699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bdd0df-eb4c-4b9b-8245-c49200a02054 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:35.998971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3005ed6-f67f-48cb-95af-50b16162e13b None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:36.004944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-b5bdd0df-eb4c-4b9b-8245-c49200a02054 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 02:05:36.004944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:36.004944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:36.004944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:36.004944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:36.004944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:36.004944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:36.004944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:36.004944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 02:05:36.004944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:36.004944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:36.004944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:36.004944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:36.004944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:36.004944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:36.004944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:36.004944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:36.005992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:36.005992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:36.005992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:36.005992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:36.005992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:36.005992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:36.005992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:36.005992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:36.005992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:36.005992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:36.005992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:36.005992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:36.005992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:36.005992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:36.005992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:36.005992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:36.006839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:36.006839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:36.006839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:36.006839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:36.006839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:36.006839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:36.006839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:36.006839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:36.006839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:36.006839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:36.006839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:36.006839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:36.006839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 02:05:36.006839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:36.006839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 503/998] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:36.007663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a0c0b-77e4-4d1d-beb6-a4adedc59506 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:36.007663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a0c0b-77e4-4d1d-beb6-a4adedc59506 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:36.007788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a0c0b-77e4-4d1d-beb6-a4adedc59506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:36.025956 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-503a0c0b-77e4-4d1d-beb6-a4adedc59506 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:36.027152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-503a0c0b-77e4-4d1d-beb6-a4adedc59506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:36.034898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bca2d-0bf3-484b-8a9e-1ea333e45b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:36.035520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bca2d-0bf3-484b-8a9e-1ea333e45b09 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:36.035798 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bca2d-0bf3-484b-8a9e-1ea333e45b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:36.036068 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bca2d-0bf3-484b-8a9e-1ea333e45b09 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:36.036783 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:36.036898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bca2d-0bf3-484b-8a9e-1ea333e45b09 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:36.038155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bca2d-0bf3-484b-8a9e-1ea333e45b09 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:36.054124 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 496/999] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:05:35 2022] POST /identity/v3/endpoints => generated 325 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:05:36.054871 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bca2d-0bf3-484b-8a9e-1ea333e45b09 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:36.076208 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:36.122375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 504/1000] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:36 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:36.146383 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a0c0b-77e4-4d1d-beb6-a4adedc59506 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:36.172050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bca2d-0bf3-484b-8a9e-1ea333e45b09 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:36.212506 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2006fd-90c5-4ebd-b95b-4625a078c36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:36.213170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2006fd-90c5-4ebd-b95b-4625a078c36a None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:36.213454 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2006fd-90c5-4ebd-b95b-4625a078c36a None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:36.213729 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2006fd-90c5-4ebd-b95b-4625a078c36a None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:36.214416 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2006fd-90c5-4ebd-b95b-4625a078c36a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:36.214729 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2006fd-90c5-4ebd-b95b-4625a078c36a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:36.216106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2006fd-90c5-4ebd-b95b-4625a078c36a None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:36.222412 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 497/1001] 10.222.0.45 () {56 vars in 1112 bytes} [Thu Aug 4 02:05:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:36.228576 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2006fd-90c5-4ebd-b95b-4625a078c36a None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:36.260994 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a0c0b-77e4-4d1d-beb6-a4adedc59506 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:36.300332 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6c5ae-8c51-4d27-94a8-f91f17252048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:36.301134 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6c5ae-8c51-4d27-94a8-f91f17252048 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:36.301423 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6c5ae-8c51-4d27-94a8-f91f17252048 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:36.301693 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6c5ae-8c51-4d27-94a8-f91f17252048 None admin] PATH_INFO: `/v3/services/image` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:36.302334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6c5ae-8c51-4d27-94a8-f91f17252048 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:36.303698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd6c5ae-8c51-4d27-94a8-f91f17252048 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:36.313445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-4dd6c5ae-8c51-4d27-94a8-f91f17252048 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 02:05:36.313445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:36.313445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:36.313445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:36.313445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:36.313445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:36.313445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:36.313445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:36.313445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 02:05:36.313445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:36.313445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:36.313445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:36.313445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:36.313445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:36.313445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:36.313445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:36.313445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:36.314497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:36.314497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:36.314497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:36.314497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:36.314497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:36.314497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:36.314497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:36.314497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:36.314497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:36.314497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:36.314497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:36.314497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:36.314497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:36.314497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:36.314497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:36.314497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:36.315978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:36.315978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:36.315978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:36.315978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:36.315978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:36.315978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:36.315978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:36.315978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:36.315978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:36.315978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:36.315978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:36.315978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:36.315978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 02:05:36.315978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:36.315978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 505/1002] 10.222.0.45 () {56 vars in 1105 bytes} [Thu Aug 4 02:05:36 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:36.321181 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6c5ae-8c51-4d27-94a8-f91f17252048 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:36.325862 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:36.329897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:36.350120 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2006fd-90c5-4ebd-b95b-4625a078c36a None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:36.386161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbca88-2891-4dfd-8639-367d224736de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:36.386701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbca88-2891-4dfd-8639-367d224736de None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:36.386984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbca88-2891-4dfd-8639-367d224736de None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:36.387258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bbca88-2891-4dfd-8639-367d224736de None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:36.387935 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbca88-2891-4dfd-8639-367d224736de None admin] RBAC: Adding query filter params (type=compute) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:36.388241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbca88-2891-4dfd-8639-367d224736de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:36.389652 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bbca88-2891-4dfd-8639-367d224736de None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:36.399306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 498/1003] 10.222.0.45 () {56 vars in 1112 bytes} [Thu Aug 4 02:05:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:36.400326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbca88-2891-4dfd-8639-367d224736de None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:36.437636 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6c5ae-8c51-4d27-94a8-f91f17252048 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:36.473672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691becc2-8310-4fa2-912a-2a083ca8a202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:36.474290 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691becc2-8310-4fa2-912a-2a083ca8a202 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:36.474565 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691becc2-8310-4fa2-912a-2a083ca8a202 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:36.474860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691becc2-8310-4fa2-912a-2a083ca8a202 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:36.475533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691becc2-8310-4fa2-912a-2a083ca8a202 None admin] RBAC: Adding query filter params (name=image) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:36.475830 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691becc2-8310-4fa2-912a-2a083ca8a202 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:36.477215 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691becc2-8310-4fa2-912a-2a083ca8a202 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:36.482393 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 506/1004] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:36 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:36.488837 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691becc2-8310-4fa2-912a-2a083ca8a202 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:36.507871 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bbca88-2891-4dfd-8639-367d224736de None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:36.550502 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dc37e-fdd1-4c77-b2ff-4e36d032ca7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:36.551185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dc37e-fdd1-4c77-b2ff-4e36d032ca7f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:36.551391 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dc37e-fdd1-4c77-b2ff-4e36d032ca7f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:36.551665 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dc37e-fdd1-4c77-b2ff-4e36d032ca7f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:36.552481 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7dc37e-fdd1-4c77-b2ff-4e36d032ca7f None admin] RBAC: Adding query filter params (interface=public, service_id=5b14470494c3492f91b19ee3c5091c1f, region_id=RegionOne) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:36.552849 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7dc37e-fdd1-4c77-b2ff-4e36d032ca7f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:36.554177 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7dc37e-fdd1-4c77-b2ff-4e36d032ca7f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:36.564303 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 499/1005] 10.222.0.45 () {56 vars in 1251 bytes} [Thu Aug 4 02:05:36 2022] GET /identity/v3/endpoints?service_id=5b14470494c3492f91b19ee3c5091c1f&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:36.566255 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dc37e-fdd1-4c77-b2ff-4e36d032ca7f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:36.570820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:36.578213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:36.608542 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691becc2-8310-4fa2-912a-2a083ca8a202 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:36.650066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95f2625-5bf1-4ec7-aaa2-683b2d4a6c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:36.650615 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f2625-5bf1-4ec7-aaa2-683b2d4a6c8f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:36.650892 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f2625-5bf1-4ec7-aaa2-683b2d4a6c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:36.651609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95f2625-5bf1-4ec7-aaa2-683b2d4a6c8f None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:36.652294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f2625-5bf1-4ec7-aaa2-683b2d4a6c8f None admin] RBAC: Adding query filter params (type=image) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:36.655602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f2625-5bf1-4ec7-aaa2-683b2d4a6c8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:36.655602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95f2625-5bf1-4ec7-aaa2-683b2d4a6c8f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:36.672659 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 507/1006] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:36 2022] GET /identity/v3/services?type=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:36.685595 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dc37e-fdd1-4c77-b2ff-4e36d032ca7f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:36.725240 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff74e38-7911-4e24-b34b-c817d42257b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:36.725919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff74e38-7911-4e24-b34b-c817d42257b8 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:36.726201 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff74e38-7911-4e24-b34b-c817d42257b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:36.726471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff74e38-7911-4e24-b34b-c817d42257b8 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:36.727134 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff74e38-7911-4e24-b34b-c817d42257b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:36.728549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff74e38-7911-4e24-b34b-c817d42257b8 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:36.744933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 500/1007] 10.222.0.45 () {56 vars in 1087 bytes} [Thu Aug 4 02:05:36 2022] GET /identity/v3/services => generated 1727 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:05:38.097961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e72227-52de-4de2-bc56-13717d49e826 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:38.098460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e72227-52de-4de2-bc56-13717d49e826 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:38.098460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e72227-52de-4de2-bc56-13717d49e826 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:38.099954 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 508/1008] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:38.105020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014479b-a098-4525-8201-636891b9cb2f None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:38.105445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014479b-a098-4525-8201-636891b9cb2f None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:38.105814 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014479b-a098-4525-8201-636891b9cb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:38.124367 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-2014479b-a098-4525-8201-636891b9cb2f None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:38.125181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2014479b-a098-4525-8201-636891b9cb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:38.196858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 501/1009] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:38 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:38.220150 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e72227-52de-4de2-bc56-13717d49e826 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:38.324985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e72227-52de-4de2-bc56-13717d49e826 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:38.354316 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:38.360722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a25936-2818-4397-82e2-6a218131557a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:38.361366 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a25936-2818-4397-82e2-6a218131557a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:38.361645 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a25936-2818-4397-82e2-6a218131557a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:38.361944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a25936-2818-4397-82e2-6a218131557a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:38.363096 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a25936-2818-4397-82e2-6a218131557a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:38.364528 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a25936-2818-4397-82e2-6a218131557a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:38.366399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 509/1010] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:38.372010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014479b-a098-4525-8201-636891b9cb2f None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:38.374221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:38.476665 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014479b-a098-4525-8201-636891b9cb2f None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:38.512027 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746bed3-6cf4-4f67-9e0a-6b4b538dd245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:38.512673 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746bed3-6cf4-4f67-9e0a-6b4b538dd245 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:38.512940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746bed3-6cf4-4f67-9e0a-6b4b538dd245 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:38.513208 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746bed3-6cf4-4f67-9e0a-6b4b538dd245 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:38.513895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746bed3-6cf4-4f67-9e0a-6b4b538dd245 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:38.515317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9746bed3-6cf4-4f67-9e0a-6b4b538dd245 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:38.542057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 502/1011] 10.222.0.45 () {60 vars in 1141 bytes} [Thu Aug 4 02:05:38 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:05:38.696233 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4f948-1380-4839-a238-c67e29c2892b None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:38.696527 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4f948-1380-4839-a238-c67e29c2892b None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:38.696835 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a4f948-1380-4839-a238-c67e29c2892b None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:38.698220 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 510/1012] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:38.703797 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d17a5-6a16-48aa-a7d5-81748d4168d5 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:38.704084 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d17a5-6a16-48aa-a7d5-81748d4168d5 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:38.704362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d17a5-6a16-48aa-a7d5-81748d4168d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:38.723443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-dc3d17a5-6a16-48aa-a7d5-81748d4168d5 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:38.724693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-dc3d17a5-6a16-48aa-a7d5-81748d4168d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:38.777130 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:38.782818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f6a22-3ff7-4dce-b086-3c6edf963617 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:38.783105 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f6a22-3ff7-4dce-b086-3c6edf963617 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:38.783387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f6a22-3ff7-4dce-b086-3c6edf963617 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:38.784825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 511/1013] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:05:38.790356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37109d2c-cf20-41e1-8de9-1914585dd4f1 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:38.790758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37109d2c-cf20-41e1-8de9-1914585dd4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:38.791112 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37109d2c-cf20-41e1-8de9-1914585dd4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:38.802118 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 503/1014] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:38 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:38.810065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-37109d2c-cf20-41e1-8de9-1914585dd4f1 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:38.811326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-37109d2c-cf20-41e1-8de9-1914585dd4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:38.825739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d17a5-6a16-48aa-a7d5-81748d4168d5 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:38.889286 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 512/1015] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:38 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 02:05:38.915879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37109d2c-cf20-41e1-8de9-1914585dd4f1 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:38.940431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d17a5-6a16-48aa-a7d5-81748d4168d5 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:38.979325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73575ad1-aeb0-45c6-9581-949a45a7ed95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:38.980234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73575ad1-aeb0-45c6-9581-949a45a7ed95 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:38.980510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73575ad1-aeb0-45c6-9581-949a45a7ed95 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:38.980818 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73575ad1-aeb0-45c6-9581-949a45a7ed95 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:38.981509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73575ad1-aeb0-45c6-9581-949a45a7ed95 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:38.982921 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73575ad1-aeb0-45c6-9581-949a45a7ed95 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:39.004692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 504/1016] 10.222.0.45 () {60 vars in 1141 bytes} [Thu Aug 4 02:05:38 2022] POST /identity/v3/services => generated 242 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:05:39.028712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37109d2c-cf20-41e1-8de9-1914585dd4f1 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:39.063851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:39.078406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0cb8b-aaef-47c7-8306-b2c746c720d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:39.079053 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0cb8b-aaef-47c7-8306-b2c746c720d4 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:39.079327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0cb8b-aaef-47c7-8306-b2c746c720d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:39.079593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0cb8b-aaef-47c7-8306-b2c746c720d4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:39.080214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0cb8b-aaef-47c7-8306-b2c746c720d4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:39.081660 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0cb8b-aaef-47c7-8306-b2c746c720d4 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:39.091707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-57a0cb8b-aaef-47c7-8306-b2c746c720d4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 02:05:39.091707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:39.091707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:39.091707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:39.091707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:39.091707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:39.091707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:39.091707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:39.091707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 02:05:39.091707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:39.091707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:39.091707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:39.091707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:39.091707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:39.091707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:39.091707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:39.091707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:39.092876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:39.092876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:39.092876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:39.092876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:39.092876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:39.092876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:39.092876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:39.092876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:39.092876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:39.092876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:39.092876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:39.092876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:39.092876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:39.092876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:39.092876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:39.092876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:39.093918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:39.093918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:39.093918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:39.093918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:39.093918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:39.093918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:39.093918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:39.093918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:39.093918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:39.093918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:39.093918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:39.093918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:39.093918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 04 02:05:39.093918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:39.093918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 513/1017] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:39.099842 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73575ad1-aeb0-45c6-9581-949a45a7ed95 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:39.152874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:39.216657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73575ad1-aeb0-45c6-9581-949a45a7ed95 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:39.250561 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf145f-e96d-4f43-a2d5-c4bcd04631ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:39.251183 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf145f-e96d-4f43-a2d5-c4bcd04631ea None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:39.251460 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf145f-e96d-4f43-a2d5-c4bcd04631ea None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:39.251730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaf145f-e96d-4f43-a2d5-c4bcd04631ea None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:39.252406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf145f-e96d-4f43-a2d5-c4bcd04631ea None admin] RBAC: Adding query filter params (name=compute) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:39.252780 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf145f-e96d-4f43-a2d5-c4bcd04631ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:39.254150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaf145f-e96d-4f43-a2d5-c4bcd04631ea None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:39.259252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 505/1018] 10.222.0.45 () {56 vars in 1112 bytes} [Thu Aug 4 02:05:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:39.265133 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0cb8b-aaef-47c7-8306-b2c746c720d4 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:39.369792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0cb8b-aaef-47c7-8306-b2c746c720d4 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:39.404110 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81374070-e335-4ef9-a27b-2d48dc3fc56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:39.404791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81374070-e335-4ef9-a27b-2d48dc3fc56c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:39.405126 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81374070-e335-4ef9-a27b-2d48dc3fc56c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:39.405406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81374070-e335-4ef9-a27b-2d48dc3fc56c None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:39.406134 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81374070-e335-4ef9-a27b-2d48dc3fc56c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:39.406442 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81374070-e335-4ef9-a27b-2d48dc3fc56c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:39.407930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81374070-e335-4ef9-a27b-2d48dc3fc56c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:39.413435 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 514/1019] 10.222.0.45 () {56 vars in 1112 bytes} [Thu Aug 4 02:05:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:39.419068 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf145f-e96d-4f43-a2d5-c4bcd04631ea None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:39.521548 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaf145f-e96d-4f43-a2d5-c4bcd04631ea None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:39.530423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:39.556448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d2800b-6878-4396-8c9a-65307ebe8c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:39.557112 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2800b-6878-4396-8c9a-65307ebe8c4f None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:39.557388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2800b-6878-4396-8c9a-65307ebe8c4f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:39.557657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d2800b-6878-4396-8c9a-65307ebe8c4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:39.558363 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2800b-6878-4396-8c9a-65307ebe8c4f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:39.559785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d2800b-6878-4396-8c9a-65307ebe8c4f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:39.585759 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 506/1020] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:05:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:05:40.614666 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e98d43-12e7-4db2-bf84-88b6442c5c7c None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:40.615226 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e98d43-12e7-4db2-bf84-88b6442c5c7c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:40.615226 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e98d43-12e7-4db2-bf84-88b6442c5c7c None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:40.616554 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 515/1021] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:40.622006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91651e-ae95-4bf2-8c0b-7bb7499b2d3c None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:40.622435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91651e-ae95-4bf2-8c0b-7bb7499b2d3c None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:40.622530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91651e-ae95-4bf2-8c0b-7bb7499b2d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:40.641156 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-bb91651e-ae95-4bf2-8c0b-7bb7499b2d3c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:40.642330 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-bb91651e-ae95-4bf2-8c0b-7bb7499b2d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:40.720753 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 507/1022] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:40.745615 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e98d43-12e7-4db2-bf84-88b6442c5c7c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:40.805564 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:40.855139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e98d43-12e7-4db2-bf84-88b6442c5c7c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:40.888723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b384f-ffec-4eeb-8824-27caf90f05c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:40.889369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b384f-ffec-4eeb-8824-27caf90f05c7 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:40.889650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b384f-ffec-4eeb-8824-27caf90f05c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:40.889925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b384f-ffec-4eeb-8824-27caf90f05c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:40.891128 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b384f-ffec-4eeb-8824-27caf90f05c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:40.892582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b384f-ffec-4eeb-8824-27caf90f05c7 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:40.901790 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 516/1023] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:40.902863 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91651e-ae95-4bf2-8c0b-7bb7499b2d3c None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:40.998195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:41.010725 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91651e-ae95-4bf2-8c0b-7bb7499b2d3c None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:41.032183 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f290b-c081-441e-af5b-62f4c213ddf0 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:41.032465 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f290b-c081-441e-af5b-62f4c213ddf0 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:41.032768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f290b-c081-441e-af5b-62f4c213ddf0 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:41.034120 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 517/1024] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:05:41.039426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b5896-946f-4866-a058-92a668c8b2f7 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:41.039711 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b5896-946f-4866-a058-92a668c8b2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:41.039982 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b5896-946f-4866-a058-92a668c8b2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:41.045866 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd7e17-29ae-416b-88fa-762d9020c2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:41.046484 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd7e17-29ae-416b-88fa-762d9020c2cc None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:41.046758 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd7e17-29ae-416b-88fa-762d9020c2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:41.047024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fd7e17-29ae-416b-88fa-762d9020c2cc None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:41.047706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd7e17-29ae-416b-88fa-762d9020c2cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:41.049161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fd7e17-29ae-416b-88fa-762d9020c2cc None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:41.058655 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-a30b5896-946f-4866-a058-92a668c8b2f7 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:41.059944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a30b5896-946f-4866-a058-92a668c8b2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:41.080663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 508/1025] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:05:40 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:05:41.147305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 518/1026] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 02:05:41.170880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd7e17-29ae-416b-88fa-762d9020c2cc None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:41.280838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fd7e17-29ae-416b-88fa-762d9020c2cc None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:41.317472 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150784ce-f6d6-4e0f-a507-b3b8ad6f6646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:41.318125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150784ce-f6d6-4e0f-a507-b3b8ad6f6646 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:41.318399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150784ce-f6d6-4e0f-a507-b3b8ad6f6646 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:41.318665 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150784ce-f6d6-4e0f-a507-b3b8ad6f6646 None admin] PATH_INFO: `/v3/services/image` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:41.319295 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150784ce-f6d6-4e0f-a507-b3b8ad6f6646 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:41.320690 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150784ce-f6d6-4e0f-a507-b3b8ad6f6646 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:41.329727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-150784ce-f6d6-4e0f-a507-b3b8ad6f6646 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 02:05:41.329727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:41.329727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:41.329727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:41.329727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:41.329727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:41.329727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:41.329727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:41.329727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 02:05:41.329727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:41.329727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:41.329727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:41.329727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:41.329727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:41.329727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:41.329727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:41.329727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:41.330829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:41.330829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:41.330829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:41.330829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:41.330829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:41.330829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:41.330829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:41.330829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:41.330829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:41.330829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:41.330829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:41.330829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:41.330829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:41.330829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:41.330829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:41.330829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:41.331678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:41.331678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:41.331678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:41.331678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:41.331678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:41.331678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:41.331678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:41.331678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:41.331678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:41.331678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:41.331678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:41.331678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:41.331678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 02:05:41.331678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:41.331678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 509/1027] 10.222.0.45 () {56 vars in 1105 bytes} [Thu Aug 4 02:05:41 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:41.337159 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b5896-946f-4866-a058-92a668c8b2f7 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:41.393169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:41.442618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b5896-946f-4866-a058-92a668c8b2f7 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:41.477248 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7764c1b-782f-4011-88c5-5f9fcd14448d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:41.477870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7764c1b-782f-4011-88c5-5f9fcd14448d None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:41.478154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7764c1b-782f-4011-88c5-5f9fcd14448d None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:41.478428 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7764c1b-782f-4011-88c5-5f9fcd14448d None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:41.479106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7764c1b-782f-4011-88c5-5f9fcd14448d None admin] RBAC: Adding query filter params (name=image) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:41.479409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7764c1b-782f-4011-88c5-5f9fcd14448d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:41.480791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7764c1b-782f-4011-88c5-5f9fcd14448d None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:41.486015 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 519/1028] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:41.491863 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150784ce-f6d6-4e0f-a507-b3b8ad6f6646 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:41.505683 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:41.598628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150784ce-f6d6-4e0f-a507-b3b8ad6f6646 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:41.634120 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d82c0-7556-45eb-92e3-cf40a05f3a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:41.635556 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d82c0-7556-45eb-92e3-cf40a05f3a95 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:41.636104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d82c0-7556-45eb-92e3-cf40a05f3a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:41.636586 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d82c0-7556-45eb-92e3-cf40a05f3a95 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:41.637512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d82c0-7556-45eb-92e3-cf40a05f3a95 None admin] RBAC: Adding query filter params (type=image) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:41.638012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d82c0-7556-45eb-92e3-cf40a05f3a95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:41.639678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d82c0-7556-45eb-92e3-cf40a05f3a95 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:41.650069 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 510/1029] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:41.651259 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7764c1b-782f-4011-88c5-5f9fcd14448d None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:41.723135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e15b-888d-468d-bc47-a64830cbf626 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:41.723341 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e15b-888d-468d-bc47-a64830cbf626 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:41.723588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c73e15b-888d-468d-bc47-a64830cbf626 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:41.725138 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 511/1030] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:05:41.729900 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dacb5-0dc2-4bfb-b24d-308b75310171 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:41.730187 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dacb5-0dc2-4bfb-b24d-308b75310171 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:41.730471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6dacb5-0dc2-4bfb-b24d-308b75310171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:41.748026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-7f6dacb5-0dc2-4bfb-b24d-308b75310171 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:41.749252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7f6dacb5-0dc2-4bfb-b24d-308b75310171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:41.762130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7764c1b-782f-4011-88c5-5f9fcd14448d None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:41.781825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:41.802992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9410f9-e156-480b-8bb3-6cb3b1084580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:41.803614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9410f9-e156-480b-8bb3-6cb3b1084580 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:41.803896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9410f9-e156-480b-8bb3-6cb3b1084580 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:41.804172 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9410f9-e156-480b-8bb3-6cb3b1084580 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:41.805402 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9410f9-e156-480b-8bb3-6cb3b1084580 None admin] RBAC: Adding query filter params (interface=public, service_id=4f3e084735844594bba5ceb53392e1fa, region_id=RegionOne) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:41.805722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9410f9-e156-480b-8bb3-6cb3b1084580 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:41.807267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9410f9-e156-480b-8bb3-6cb3b1084580 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:41.817676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 520/1031] 10.222.0.45 () {56 vars in 1251 bytes} [Thu Aug 4 02:05:41 2022] GET /identity/v3/endpoints?service_id=4f3e084735844594bba5ceb53392e1fa&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:41.818743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9410f9-e156-480b-8bb3-6cb3b1084580 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:41.823774 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 512/1032] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:41 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 02:05:41.847263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6dacb5-0dc2-4bfb-b24d-308b75310171 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:41.850362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:41.929340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9410f9-e156-480b-8bb3-6cb3b1084580 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:41.951549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6dacb5-0dc2-4bfb-b24d-308b75310171 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:41.966499 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46d0c1-8bd3-42f7-bb33-d24b8079d85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:41.967071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46d0c1-8bd3-42f7-bb33-d24b8079d85a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:41.967353 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46d0c1-8bd3-42f7-bb33-d24b8079d85a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:41.967628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46d0c1-8bd3-42f7-bb33-d24b8079d85a None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:41.968277 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46d0c1-8bd3-42f7-bb33-d24b8079d85a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:41.969682 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46d0c1-8bd3-42f7-bb33-d24b8079d85a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:41.984744 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 521/1033] 10.222.0.45 () {56 vars in 1087 bytes} [Thu Aug 4 02:05:41 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:05:41.987291 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec0c28-5c98-4bbf-82e8-9964610ca377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:41.987961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec0c28-5c98-4bbf-82e8-9964610ca377 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:41.988244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec0c28-5c98-4bbf-82e8-9964610ca377 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:41.988518 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec0c28-5c98-4bbf-82e8-9964610ca377 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:42.001045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec0c28-5c98-4bbf-82e8-9964610ca377 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:05:42.006755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-1eec0c28-5c98-4bbf-82e8-9964610ca377 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 02:05:42.006755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:42.006755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:42.006755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:42.006755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:42.006755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:42.006755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:42.006755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:42.006755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:42.006755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:42.006755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:42.006755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:42.006755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:05:42.006755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:05:42.006755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:05:42.006755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:05:42.007769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:05:42.007769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:05:42.007769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:05:42.007769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:05:42.007769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:42.007769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:42.007769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:42.007769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:42.007769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:42.007769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:42.007769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:42.007769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:42.007769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:42.007769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:42.007769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:42.007769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:42.008762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:42.008762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:42.008762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:42.008762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:42.008762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:05:42.008762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:05:42.008762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:05:42.008762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:05:42.008762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:05:42.008762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:05:42.008762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 02:05:42.008762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:42.013173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 513/1034] 10.222.0.45 () {56 vars in 1120 bytes} [Thu Aug 4 02:05:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:42.014477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46d0c1-8bd3-42f7-bb33-d24b8079d85a None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:42.066999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:42.126285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46d0c1-8bd3-42f7-bb33-d24b8079d85a None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:42.165427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2c3b5-bbc2-4c83-8ce1-1257cc549a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:42.166049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2c3b5-bbc2-4c83-8ce1-1257cc549a04 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:42.166329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2c3b5-bbc2-4c83-8ce1-1257cc549a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:42.166603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f2c3b5-bbc2-4c83-8ce1-1257cc549a04 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:42.167321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2c3b5-bbc2-4c83-8ce1-1257cc549a04 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:42.167624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2c3b5-bbc2-4c83-8ce1-1257cc549a04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:42.169014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f2c3b5-bbc2-4c83-8ce1-1257cc549a04 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:42.177853 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 522/1035] 10.222.0.45 () {56 vars in 1115 bytes} [Thu Aug 4 02:05:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:42.183520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec0c28-5c98-4bbf-82e8-9964610ca377 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:42.252273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:42.293628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec0c28-5c98-4bbf-82e8-9964610ca377 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:42.329013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20954f25-5db2-4074-abb3-d6d55c5a3f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:42.329648 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20954f25-5db2-4074-abb3-d6d55c5a3f09 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:42.329924 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20954f25-5db2-4074-abb3-d6d55c5a3f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:42.330194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20954f25-5db2-4074-abb3-d6d55c5a3f09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:42.340519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20954f25-5db2-4074-abb3-d6d55c5a3f09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:42.342002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20954f25-5db2-4074-abb3-d6d55c5a3f09 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:42.356588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-20954f25-5db2-4074-abb3-d6d55c5a3f09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:42.356588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:42.356588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:42.356588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:42.356588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:42.356588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:42.356588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:42.356588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:42.356588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:42.356588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:42.356588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:42.356588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:42.356588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:42.356588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:42.356588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:42.357633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:42.357633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:42.357633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:42.357633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:42.357633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:42.357633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:42.357633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:42.357633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:42.357633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:42.357633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:42.357633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:42.357633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:42.357633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:42.357633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:42.357633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:42.357633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:42.357633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:42.358525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:42.358525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:42.358525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:42.358525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:42.360704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 514/1036] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:42.364286 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2c3b5-bbc2-4c83-8ce1-1257cc549a04 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:42.472216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f2c3b5-bbc2-4c83-8ce1-1257cc549a04 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:42.479486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:42.510701 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d214a6-b3d7-4c3f-b230-1279a477027c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:42.511223 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d214a6-b3d7-4c3f-b230-1279a477027c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:42.511501 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d214a6-b3d7-4c3f-b230-1279a477027c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:42.511768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d214a6-b3d7-4c3f-b230-1279a477027c None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:42.512456 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d214a6-b3d7-4c3f-b230-1279a477027c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:42.512776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d214a6-b3d7-4c3f-b230-1279a477027c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:42.514132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d214a6-b3d7-4c3f-b230-1279a477027c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:42.530971 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 523/1037] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:42.532592 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20954f25-5db2-4074-abb3-d6d55c5a3f09 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:42.632918 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20954f25-5db2-4074-abb3-d6d55c5a3f09 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:42.650160 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:42.669672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a930b-59ee-4ef1-81cf-57fddd82a51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:42.670320 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a930b-59ee-4ef1-81cf-57fddd82a51f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:42.670602 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a930b-59ee-4ef1-81cf-57fddd82a51f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:42.670876 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a930b-59ee-4ef1-81cf-57fddd82a51f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:42.679667 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a930b-59ee-4ef1-81cf-57fddd82a51f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:42.681161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a930b-59ee-4ef1-81cf-57fddd82a51f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:42.692751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-0c8a930b-59ee-4ef1-81cf-57fddd82a51f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:05:42.692751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:42.692751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:42.692751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:42.692751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:42.692751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:42.692751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:42.692751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:42.692751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:42.692751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:42.692751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:42.692751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:42.692751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:05:42.692751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:05:42.692751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:05:42.692751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:42.693692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:42.695209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:42.695209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:42.695209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:05:42.695209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:05:42.695209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:05:42.695209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:05:42.695209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:05:42.695209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:05:42.695209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:05:42.695209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:42.695209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 515/1038] 10.222.0.45 () {56 vars in 1093 bytes} [Thu Aug 4 02:05:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:42.700079 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d214a6-b3d7-4c3f-b230-1279a477027c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:42.803115 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d214a6-b3d7-4c3f-b230-1279a477027c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:42.837063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708b45d-edc4-42bc-81a3-350da39d581f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:42.837679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708b45d-edc4-42bc-81a3-350da39d581f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:42.837962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708b45d-edc4-42bc-81a3-350da39d581f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:42.838240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708b45d-edc4-42bc-81a3-350da39d581f None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:42.839031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8708b45d-edc4-42bc-81a3-350da39d581f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:42.839339 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8708b45d-edc4-42bc-81a3-350da39d581f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:42.840792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8708b45d-edc4-42bc-81a3-350da39d581f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:42.863792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 524/1039] 10.222.0.45 () {56 vars in 1133 bytes} [Thu Aug 4 02:05:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:42.865276 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a930b-59ee-4ef1-81cf-57fddd82a51f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:42.973774 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a930b-59ee-4ef1-81cf-57fddd82a51f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:43.006945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11feda65-546e-46e3-adc5-88ab309fb46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:43.007564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11feda65-546e-46e3-adc5-88ab309fb46b None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:43.007845 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11feda65-546e-46e3-adc5-88ab309fb46b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:43.008161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11feda65-546e-46e3-adc5-88ab309fb46b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:43.054238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:43.078804 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11feda65-546e-46e3-adc5-88ab309fb46b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:43.080292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11feda65-546e-46e3-adc5-88ab309fb46b None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:43.187657 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d71f03-6c73-4cc1-a100-70aff4450b57 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:43.187854 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 516/1040] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:05:42 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:05:43.187975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d71f03-6c73-4cc1-a100-70aff4450b57 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:43.187975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d71f03-6c73-4cc1-a100-70aff4450b57 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:43.187975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 525/1041] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:05:43.189743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11feda65-546e-46e3-adc5-88ab309fb46b None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:43.190757 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9f257-a9e2-4812-b293-49035415c91e None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:43.191034 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9f257-a9e2-4812-b293-49035415c91e None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:43.191304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9f257-a9e2-4812-b293-49035415c91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:43.210091 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-98b9f257-a9e2-4812-b293-49035415c91e None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:43.211248 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-98b9f257-a9e2-4812-b293-49035415c91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:43.229052 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:43.279562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:43.284666 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 526/1042] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:43 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 02:05:43.306000 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9f257-a9e2-4812-b293-49035415c91e None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:43.316890 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11feda65-546e-46e3-adc5-88ab309fb46b None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:43.352998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc64483-6dbb-4d27-b7c7-b4540c9d81cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:43.353752 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc64483-6dbb-4d27-b7c7-b4540c9d81cc None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:43.354015 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc64483-6dbb-4d27-b7c7-b4540c9d81cc None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:43.354181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc64483-6dbb-4d27-b7c7-b4540c9d81cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:43.365180 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc64483-6dbb-4d27-b7c7-b4540c9d81cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:43.366615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc64483-6dbb-4d27-b7c7-b4540c9d81cc None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:43.381696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-acc64483-6dbb-4d27-b7c7-b4540c9d81cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:43.381696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:43.381696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:43.381696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:43.381696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:43.381696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:43.381696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:43.381696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:43.381696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:43.381696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:43.381696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:43.381696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:43.381696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:43.381696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:43.381696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:43.382642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:43.382642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:43.382642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:43.382642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:43.382642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:43.382642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:43.382642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:43.382642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:43.382642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:43.382642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:43.382642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:43.382642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:43.382642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:43.382642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:43.382642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:43.382642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:43.382642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:43.383568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:43.383568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:43.383568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:43.383568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:43.387971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 517/1043] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:43.388722 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc64483-6dbb-4d27-b7c7-b4540c9d81cc None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:43.417559 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9f257-a9e2-4812-b293-49035415c91e None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:43.456262 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4271050-0b84-4f15-95ba-e571b1fa8acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:43.456934 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4271050-0b84-4f15-95ba-e571b1fa8acc None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:43.457219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4271050-0b84-4f15-95ba-e571b1fa8acc None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:43.457494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4271050-0b84-4f15-95ba-e571b1fa8acc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:43.469919 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4271050-0b84-4f15-95ba-e571b1fa8acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:05:43.476202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-d4271050-0b84-4f15-95ba-e571b1fa8acc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:05:43.476202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:43.476202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:43.476202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:43.476202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:43.476202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:43.476202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:43.476202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:43.476202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:43.476202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:43.476202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:43.476202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:43.476202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:05:43.476202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:05:43.476202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:05:43.476202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:05:43.477417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:05:43.477417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:05:43.477417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:05:43.477417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:05:43.477417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:43.477417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:43.477417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:43.477417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:43.477417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:43.477417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:43.477417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:43.477417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:43.477417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:43.477417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:43.477417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:43.477417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:43.478464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:43.478464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:43.478464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:43.478464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:43.478464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:05:43.478464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:05:43.478464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:05:43.478464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:05:43.478464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:05:43.478464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:05:43.478464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:05:43.478464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:43.479173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 527/1044] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:05:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:43.485787 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4271050-0b84-4f15-95ba-e571b1fa8acc None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:43.508726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc64483-6dbb-4d27-b7c7-b4540c9d81cc None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:43.511095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:43.542947 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:43.552694 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223815e-650f-44da-8982-2c834e4812c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:43.553363 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223815e-650f-44da-8982-2c834e4812c6 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:43.553594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223815e-650f-44da-8982-2c834e4812c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:43.553872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223815e-650f-44da-8982-2c834e4812c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:43.554568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223815e-650f-44da-8982-2c834e4812c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:43.554874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223815e-650f-44da-8982-2c834e4812c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:43.556321 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223815e-650f-44da-8982-2c834e4812c6 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:43.568581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 518/1045] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:43.574086 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223815e-650f-44da-8982-2c834e4812c6 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:43.615396 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4271050-0b84-4f15-95ba-e571b1fa8acc None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:43.660255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc89e40-1386-4c84-a7bf-4c6debbdc65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:43.661274 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc89e40-1386-4c84-a7bf-4c6debbdc65c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:43.661562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc89e40-1386-4c84-a7bf-4c6debbdc65c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:43.662163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc89e40-1386-4c84-a7bf-4c6debbdc65c None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:43.662899 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc89e40-1386-4c84-a7bf-4c6debbdc65c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:43.663257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc89e40-1386-4c84-a7bf-4c6debbdc65c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:43.664645 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc89e40-1386-4c84-a7bf-4c6debbdc65c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:43.678512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 528/1046] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:05:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:43.679691 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc89e40-1386-4c84-a7bf-4c6debbdc65c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:43.690240 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223815e-650f-44da-8982-2c834e4812c6 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:43.728027 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092d94b-e7c7-4d06-b8d5-3a80bf20e6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:43.728873 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092d94b-e7c7-4d06-b8d5-3a80bf20e6ab None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:43.729155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092d94b-e7c7-4d06-b8d5-3a80bf20e6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:43.729436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092d94b-e7c7-4d06-b8d5-3a80bf20e6ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:43.736530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092d94b-e7c7-4d06-b8d5-3a80bf20e6ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:43.738260 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092d94b-e7c7-4d06-b8d5-3a80bf20e6ab None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:43.750106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-4092d94b-e7c7-4d06-b8d5-3a80bf20e6ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:43.750106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:43.750106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:43.750106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:43.750106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:43.750106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:43.750106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:43.750106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:43.750106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:43.750106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:43.750106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:43.750106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:43.750106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:05:43.750106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:05:43.750106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:05:43.751057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:05:43.751057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:43.751057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:43.751057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:43.751057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:43.751057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:43.751057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:43.751057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:43.751057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:43.751057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:43.751057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:43.751057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:43.751057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:43.751057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:43.751057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:43.751057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:43.751057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:43.751959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:05:43.751959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:05:43.751959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:05:43.751959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:05:43.751959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:05:43.751959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:05:43.751959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:43.751959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:43.751959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 519/1047] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:43.757490 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092d94b-e7c7-4d06-b8d5-3a80bf20e6ab None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:43.762805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:43.793601 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc89e40-1386-4c84-a7bf-4c6debbdc65c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:43.812855 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:43.830322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb7c9f-f12f-4891-9f2b-81f815b39b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:43.831482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb7c9f-f12f-4891-9f2b-81f815b39b51 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:43.831766 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb7c9f-f12f-4891-9f2b-81f815b39b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:43.832042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb7c9f-f12f-4891-9f2b-81f815b39b51 None admin] PATH_INFO: `/v3/users/636376a57aad401fb3cd7ca83557bb86` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:43.848489 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb7c9f-f12f-4891-9f2b-81f815b39b51 None admin] RBAC: Authorizing `identity:get_user(user_id=636376a57aad401fb3cd7ca83557bb86)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:43.850148 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb7c9f-f12f-4891-9f2b-81f815b39b51 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:43.856568 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 529/1048] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:05:43 2022] GET /identity/v3/users/636376a57aad401fb3cd7ca83557bb86 => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:43.857526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb7c9f-f12f-4891-9f2b-81f815b39b51 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:43.872009 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092d94b-e7c7-4d06-b8d5-3a80bf20e6ab None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:43.910656 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0faad-495a-4111-9c4c-d8c391ca50f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:43.911280 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0faad-495a-4111-9c4c-d8c391ca50f1 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:43.911556 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0faad-495a-4111-9c4c-d8c391ca50f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:43.911929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0faad-495a-4111-9c4c-d8c391ca50f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:43.912576 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0faad-495a-4111-9c4c-d8c391ca50f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:43.912912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0faad-495a-4111-9c4c-d8c391ca50f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:43.914340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c0faad-495a-4111-9c4c-d8c391ca50f1 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:43.927976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 520/1049] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:05:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:43.933875 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0faad-495a-4111-9c4c-d8c391ca50f1 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:43.966747 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb7c9f-f12f-4891-9f2b-81f815b39b51 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:44.002861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc15869-4c28-42c7-be3a-ada4661f61a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:44.003472 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc15869-4c28-42c7-be3a-ada4661f61a8 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:44.003754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc15869-4c28-42c7-be3a-ada4661f61a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:44.004023 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc15869-4c28-42c7-be3a-ada4661f61a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:44.006640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:44.040414 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0faad-495a-4111-9c4c-d8c391ca50f1 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:44.050668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:44.074940 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc15869-4c28-42c7-be3a-ada4661f61a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:44.076129 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c15025-bd69-45d5-a5a6-4554039ab764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:44.076297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc15869-4c28-42c7-be3a-ada4661f61a8 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:44.076769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c15025-bd69-45d5-a5a6-4554039ab764 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:44.077048 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c15025-bd69-45d5-a5a6-4554039ab764 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:44.077320 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c15025-bd69-45d5-a5a6-4554039ab764 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:44.078095 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c15025-bd69-45d5-a5a6-4554039ab764 None admin] RBAC: Adding query filter params (role.id=609fcaecc1a648f8b41c081fe0703ef4, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=c571406d2a3f40dab035a918399d6f6a) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:44.078397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c15025-bd69-45d5-a5a6-4554039ab764 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:44.079813 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c15025-bd69-45d5-a5a6-4554039ab764 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:44.086375 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 521/1050] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:05:43 2022] GET /identity/v3/role_assignments?user.id=c571406d2a3f40dab035a918399d6f6a&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=609fcaecc1a648f8b41c081fe0703ef4 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:44.087378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21c25c-02b7-4882-94f7-22953f542dee None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:44.087658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21c25c-02b7-4882-94f7-22953f542dee None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:44.087928 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21c25c-02b7-4882-94f7-22953f542dee None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:44.089337 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 522/1051] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:44.094868 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f308b3-9e9a-4562-a78b-ac224a8b7e85 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:44.095146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f308b3-9e9a-4562-a78b-ac224a8b7e85 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:44.095418 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f308b3-9e9a-4562-a78b-ac224a8b7e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:44.115807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-25f308b3-9e9a-4562-a78b-ac224a8b7e85 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:44.117109 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-25f308b3-9e9a-4562-a78b-ac224a8b7e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:44.196507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 523/1052] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 02:05:44.217867 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 530/1053] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:05:43 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 02:05:44.220715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f308b3-9e9a-4562-a78b-ac224a8b7e85 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:44.223478 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc15869-4c28-42c7-be3a-ada4661f61a8 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:44.260245 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:44.336006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f308b3-9e9a-4562-a78b-ac224a8b7e85 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:44.337708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:44.344410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc15869-4c28-42c7-be3a-ada4661f61a8 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:44.373058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377dabf-6511-48a6-bf8b-16f068e887ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:44.373882 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377dabf-6511-48a6-bf8b-16f068e887ee None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:44.374209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377dabf-6511-48a6-bf8b-16f068e887ee None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:44.374544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c377dabf-6511-48a6-bf8b-16f068e887ee None admin] PATH_INFO: `/v3/services/image` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:44.375162 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377dabf-6511-48a6-bf8b-16f068e887ee None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:44.376513 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c377dabf-6511-48a6-bf8b-16f068e887ee None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:44.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-c377dabf-6511-48a6-bf8b-16f068e887ee None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 02:05:44.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:44.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:05:44.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:05:44.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:05:44.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:05:44.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:05:44.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:44.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 02:05:44.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:44.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:05:44.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:44.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:44.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:44.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:44.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:44.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:44.387658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:44.387658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:44.387658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:44.387658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:44.387658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:44.387658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:44.387658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:05:44.387658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:05:44.387658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:05:44.387658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:05:44.387658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:44.387658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:44.387658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:44.387658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:44.387658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:44.387658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:44.389521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:44.389521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:44.389521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:44.389521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:44.389521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:44.389521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:44.389521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:44.389521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:44.389521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:44.389521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:44.389521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:05:44.389521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:05:44.389521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 04 02:05:44.389521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:44.392496 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 524/1054] 10.222.0.45 () {56 vars in 1105 bytes} [Thu Aug 4 02:05:44 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:44.394123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8b146-a371-45bf-b80c-fbc8dacec1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:44.394347 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377dabf-6511-48a6-bf8b-16f068e887ee None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:44.395824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8b146-a371-45bf-b80c-fbc8dacec1ce None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:44.396553 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8b146-a371-45bf-b80c-fbc8dacec1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:44.397839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c8b146-a371-45bf-b80c-fbc8dacec1ce None admin] PATH_INFO: `/v3/projects/f2215af9cfee46539b78848b38d40863` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:44.400355 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c8b146-a371-45bf-b80c-fbc8dacec1ce None admin] RBAC: Authorizing `identity:get_project(project_id=f2215af9cfee46539b78848b38d40863)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:44.402071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c8b146-a371-45bf-b80c-fbc8dacec1ce None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:44.404086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 531/1055] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:05:44 2022] GET /identity/v3/projects/f2215af9cfee46539b78848b38d40863 => generated 311 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:44.410511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8b146-a371-45bf-b80c-fbc8dacec1ce None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:44.518734 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c377dabf-6511-48a6-bf8b-16f068e887ee None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:44.520859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c8b146-a371-45bf-b80c-fbc8dacec1ce None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:44.522860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:44.567302 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b7e82-56f9-4a6c-9402-c83ff4ea5093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:44.567873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76dda48-ca87-48c7-9913-3ed385bf1da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:44.568216 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b7e82-56f9-4a6c-9402-c83ff4ea5093 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:44.568669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b7e82-56f9-4a6c-9402-c83ff4ea5093 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:44.568997 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dda48-ca87-48c7-9913-3ed385bf1da1 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:44.569110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b7e82-56f9-4a6c-9402-c83ff4ea5093 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:44.569509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dda48-ca87-48c7-9913-3ed385bf1da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:44.569926 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76dda48-ca87-48c7-9913-3ed385bf1da1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:44.570141 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b7e82-56f9-4a6c-9402-c83ff4ea5093 None admin] RBAC: Adding query filter params (name=image) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:44.570454 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b7e82-56f9-4a6c-9402-c83ff4ea5093 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:44.571175 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76dda48-ca87-48c7-9913-3ed385bf1da1 None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=f2215af9cfee46539b78848b38d40863, user.id=636376a57aad401fb3cd7ca83557bb86) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:44.571628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76dda48-ca87-48c7-9913-3ed385bf1da1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:44.571826 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b7e82-56f9-4a6c-9402-c83ff4ea5093 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:44.573342 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76dda48-ca87-48c7-9913-3ed385bf1da1 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:44.577515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 525/1056] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:44.579807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 532/1057] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:05:44 2022] GET /identity/v3/role_assignments?user.id=636376a57aad401fb3cd7ca83557bb86&scope.project.id=f2215af9cfee46539b78848b38d40863&role.id=35205394c8af4c52a61b12f92ba16644 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:44.593256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b7e82-56f9-4a6c-9402-c83ff4ea5093 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:44.628474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:44.718987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b7e82-56f9-4a6c-9402-c83ff4ea5093 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:44.755438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7c141-d1e5-4124-8e2f-631c448c0b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:44.756042 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7c141-d1e5-4124-8e2f-631c448c0b04 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:44.756318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7c141-d1e5-4124-8e2f-631c448c0b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:44.756585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7c141-d1e5-4124-8e2f-631c448c0b04 None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:44.757282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7c141-d1e5-4124-8e2f-631c448c0b04 None admin] RBAC: Adding query filter params (type=image) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:44.757582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7c141-d1e5-4124-8e2f-631c448c0b04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:44.758945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7c141-d1e5-4124-8e2f-631c448c0b04 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:44.764044 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 526/1058] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:44.770183 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76dda48-ca87-48c7-9913-3ed385bf1da1 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:44.876061 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76dda48-ca87-48c7-9913-3ed385bf1da1 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:44.911236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a0367-1ce2-47c1-910f-576423fdea9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:44.911852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a0367-1ce2-47c1-910f-576423fdea9a None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:44.912137 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a0367-1ce2-47c1-910f-576423fdea9a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:44.912407 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a0367-1ce2-47c1-910f-576423fdea9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:44.913152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a0367-1ce2-47c1-910f-576423fdea9a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:44.914572 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935a0367-1ce2-47c1-910f-576423fdea9a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:44.938083 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 533/1059] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:05:44 2022] POST /identity/v3/endpoints => generated 327 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:05:46.117629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31064de5-b584-4b58-8ace-e682ccb92beb None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:46.118128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31064de5-b584-4b58-8ace-e682ccb92beb None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:46.118128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31064de5-b584-4b58-8ace-e682ccb92beb None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:46.119487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 527/1060] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:46.124635 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19db6-7189-4cd2-8d48-dcda2baee1a9 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:46.125056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19db6-7189-4cd2-8d48-dcda2baee1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:46.125239 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19db6-7189-4cd2-8d48-dcda2baee1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:46.143734 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-29b19db6-7189-4cd2-8d48-dcda2baee1a9 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:46.144957 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-29b19db6-7189-4cd2-8d48-dcda2baee1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:46.161844 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:46.219999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 534/1061] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:46.242623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31064de5-b584-4b58-8ace-e682ccb92beb None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:46.339448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:46.349025 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31064de5-b584-4b58-8ace-e682ccb92beb None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:46.383054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f17ec3-9805-4592-8085-c579be470a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:46.383704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f17ec3-9805-4592-8085-c579be470a31 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:46.383993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f17ec3-9805-4592-8085-c579be470a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:46.384262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f17ec3-9805-4592-8085-c579be470a31 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:46.395712 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f17ec3-9805-4592-8085-c579be470a31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:05:46.400536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-63f17ec3-9805-4592-8085-c579be470a31 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 02:05:46.400536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:46.400536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:46.400536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:46.400536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:46.400536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:46.400536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:46.400536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:46.400536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:46.400536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:46.400536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:46.400536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:46.400536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:05:46.400536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:05:46.400536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:05:46.400536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:05:46.401530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:05:46.401530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:05:46.401530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:05:46.401530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:05:46.401530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:46.401530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:46.401530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:46.401530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:46.401530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:46.401530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:46.401530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:46.401530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:46.401530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:46.401530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:46.401530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:46.401530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:46.402394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:46.402394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:46.402394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:46.402394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:46.402394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:05:46.402394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:05:46.402394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:05:46.402394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:05:46.402394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:05:46.402394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:05:46.402394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 02:05:46.402394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:46.403686 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 528/1062] 10.222.0.45 () {56 vars in 1120 bytes} [Thu Aug 4 02:05:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:46.408011 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b19db6-7189-4cd2-8d48-dcda2baee1a9 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:46.509773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b19db6-7189-4cd2-8d48-dcda2baee1a9 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:46.544413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a25bd-f8c0-469b-b1b7-eecf65e868ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:46.545190 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a25bd-f8c0-469b-b1b7-eecf65e868ea None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:46.545378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a25bd-f8c0-469b-b1b7-eecf65e868ea None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:46.545621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a25bd-f8c0-469b-b1b7-eecf65e868ea None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:46.546342 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a25bd-f8c0-469b-b1b7-eecf65e868ea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:46.546648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a25bd-f8c0-469b-b1b7-eecf65e868ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:46.548007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a25bd-f8c0-469b-b1b7-eecf65e868ea None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:46.557153 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 535/1063] 10.222.0.45 () {56 vars in 1115 bytes} [Thu Aug 4 02:05:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:46.563091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f17ec3-9805-4592-8085-c579be470a31 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:46.595055 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb0cd0-7336-40cd-aee1-51845f84af2a None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:46.595361 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb0cd0-7336-40cd-aee1-51845f84af2a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:46.595606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bb0cd0-7336-40cd-aee1-51845f84af2a None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:46.597554 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 536/1064] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:05:46.603129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd34e0f-3427-472f-b31d-3e098dba6929 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:46.603406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd34e0f-3427-472f-b31d-3e098dba6929 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:46.603673 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd34e0f-3427-472f-b31d-3e098dba6929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:46.623400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-1bd34e0f-3427-472f-b31d-3e098dba6929 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:46.624580 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1bd34e0f-3427-472f-b31d-3e098dba6929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:46.627790 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:46.676587 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f17ec3-9805-4592-8085-c579be470a31 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:46.707816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 537/1065] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:46.712173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50546863-8662-4091-a914-d0f31af8b9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:46.712831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50546863-8662-4091-a914-d0f31af8b9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:46.713107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50546863-8662-4091-a914-d0f31af8b9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:46.713373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50546863-8662-4091-a914-d0f31af8b9e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:46.724547 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50546863-8662-4091-a914-d0f31af8b9e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:46.726069 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50546863-8662-4091-a914-d0f31af8b9e8 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:46.730398 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd34e0f-3427-472f-b31d-3e098dba6929 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:46.741641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-50546863-8662-4091-a914-d0f31af8b9e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:46.741641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:46.741641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:46.741641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:46.741641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:46.741641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:46.741641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:46.741641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:46.741641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:46.741641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:46.741641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:46.741641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:46.741641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:46.741641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:46.741641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:46.742599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:46.742599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:46.742599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:46.742599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:46.742599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:46.742599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:46.742599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:46.742599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:46.742599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:46.742599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:46.742599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:46.742599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:46.742599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:46.742599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:46.742599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:46.742599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:46.742599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:46.743501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:46.743501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:46.743501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:46.743501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:46.743501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 529/1066] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:46.748399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50546863-8662-4091-a914-d0f31af8b9e8 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:46.758044 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:46.847031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd34e0f-3427-472f-b31d-3e098dba6929 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:46.855952 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50546863-8662-4091-a914-d0f31af8b9e8 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:46.884413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f885ae-4e9f-4f14-a84e-d1c373f9142c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:46.885091 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f885ae-4e9f-4f14-a84e-d1c373f9142c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:46.885368 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f885ae-4e9f-4f14-a84e-d1c373f9142c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:46.885634 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f885ae-4e9f-4f14-a84e-d1c373f9142c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:46.890474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95036d5e-2f4a-440d-a497-5ee0316c8687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:46.891109 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95036d5e-2f4a-440d-a497-5ee0316c8687 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:46.891362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95036d5e-2f4a-440d-a497-5ee0316c8687 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:46.891629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95036d5e-2f4a-440d-a497-5ee0316c8687 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:46.892314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95036d5e-2f4a-440d-a497-5ee0316c8687 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:46.896841 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95036d5e-2f4a-440d-a497-5ee0316c8687 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:46.896841 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95036d5e-2f4a-440d-a497-5ee0316c8687 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:46.900242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:46.904419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f885ae-4e9f-4f14-a84e-d1c373f9142c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:05:46.911076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-48f885ae-4e9f-4f14-a84e-d1c373f9142c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:05:46.911076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:46.911076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:46.911076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:46.911076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:46.911076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:46.911076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:46.911076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:46.911076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:46.911076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:46.911076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:46.911076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:46.911076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:05:46.911076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:05:46.911076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:05:46.911076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:05:46.912063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:05:46.912063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:05:46.912063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:05:46.912063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:05:46.912063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:46.912063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:46.912063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:46.912063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:46.912063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:46.912063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:46.912063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:46.912063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:46.912063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:46.912063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:46.912063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:46.912063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:46.912955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:46.912955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:46.912955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:46.912955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:46.912955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:05:46.912955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:05:46.912955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:05:46.912955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:05:46.912955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:05:46.912955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:05:46.912955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:05:46.912955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:46.913597 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 530/1067] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:46.922289 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95036d5e-2f4a-440d-a497-5ee0316c8687 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:46.922790 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 538/1068] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:05:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:46.923695 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f885ae-4e9f-4f14-a84e-d1c373f9142c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:47.000533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:47.037237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95036d5e-2f4a-440d-a497-5ee0316c8687 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:47.040998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f885ae-4e9f-4f14-a84e-d1c373f9142c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:47.071622 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e345b9e-9289-4ea7-aa66-311a0548b535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:47.072233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e345b9e-9289-4ea7-aa66-311a0548b535 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:47.072510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e345b9e-9289-4ea7-aa66-311a0548b535 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:47.072809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e345b9e-9289-4ea7-aa66-311a0548b535 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:47.073518 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e345b9e-9289-4ea7-aa66-311a0548b535 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:47.073820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e345b9e-9289-4ea7-aa66-311a0548b535 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:47.075187 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e345b9e-9289-4ea7-aa66-311a0548b535 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:47.078075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0956b61-872a-46c1-ac9c-7eb8399de21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:47.078710 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0956b61-872a-46c1-ac9c-7eb8399de21c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:47.078993 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0956b61-872a-46c1-ac9c-7eb8399de21c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:47.079270 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0956b61-872a-46c1-ac9c-7eb8399de21c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:47.083342 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 531/1069] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:05:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:47.084524 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a085b8e-6138-4072-a381-84ead0f79714 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:47.084830 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a085b8e-6138-4072-a381-84ead0f79714 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:47.085102 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a085b8e-6138-4072-a381-84ead0f79714 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:47.086482 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 532/1070] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:47.088020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0956b61-872a-46c1-ac9c-7eb8399de21c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:47.089488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0956b61-872a-46c1-ac9c-7eb8399de21c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:47.089709 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a085b8e-6138-4072-a381-84ead0f79714 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:47.101931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-d0956b61-872a-46c1-ac9c-7eb8399de21c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:05:47.101931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:47.101931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:47.101931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:47.101931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:47.101931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:47.101931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:47.101931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:47.101931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:47.101931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:47.101931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:47.101931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:47.101931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:05:47.101931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:05:47.101931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:05:47.101931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:47.102960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:47.104491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:47.104491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:47.104491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:05:47.104491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:05:47.104491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:05:47.104491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:05:47.104491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:05:47.104491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:05:47.104491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:05:47.104491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:47.104491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 539/1071] 10.222.0.45 () {56 vars in 1093 bytes} [Thu Aug 4 02:05:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:47.105271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec09533-7f2d-40f1-bf5c-b3959193ad3c None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:47.105271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec09533-7f2d-40f1-bf5c-b3959193ad3c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:47.105271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec09533-7f2d-40f1-bf5c-b3959193ad3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:47.122662 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-7ec09533-7f2d-40f1-bf5c-b3959193ad3c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:47.123855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7ec09533-7f2d-40f1-bf5c-b3959193ad3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:47.179762 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:47.192026 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 540/1072] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:47.193092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec09533-7f2d-40f1-bf5c-b3959193ad3c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:47.197478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a085b8e-6138-4072-a381-84ead0f79714 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:47.225540 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:47.234449 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9b1dc-277f-48e1-813c-063dc7137a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:47.235089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9b1dc-277f-48e1-813c-063dc7137a67 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:47.235371 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9b1dc-277f-48e1-813c-063dc7137a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:47.235647 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9b1dc-277f-48e1-813c-063dc7137a67 None admin] PATH_INFO: `/v3/users/636376a57aad401fb3cd7ca83557bb86` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:47.236962 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9b1dc-277f-48e1-813c-063dc7137a67 None admin] RBAC: Authorizing `identity:get_user(user_id=636376a57aad401fb3cd7ca83557bb86)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:47.238901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de9b1dc-277f-48e1-813c-063dc7137a67 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:47.240512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 533/1073] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:05:47 2022] GET /identity/v3/users/636376a57aad401fb3cd7ca83557bb86 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:47.241293 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9b1dc-277f-48e1-813c-063dc7137a67 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:47.305334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec09533-7f2d-40f1-bf5c-b3959193ad3c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:47.339881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753f852-ed84-458f-806d-0dfc7e1dc108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:47.340496 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753f852-ed84-458f-806d-0dfc7e1dc108 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:47.340884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753f852-ed84-458f-806d-0dfc7e1dc108 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:47.341066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6753f852-ed84-458f-806d-0dfc7e1dc108 None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:47.341857 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753f852-ed84-458f-806d-0dfc7e1dc108 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:47.342169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753f852-ed84-458f-806d-0dfc7e1dc108 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:47.343583 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6753f852-ed84-458f-806d-0dfc7e1dc108 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:47.344889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9b1dc-277f-48e1-813c-063dc7137a67 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:47.360782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 541/1074] 10.222.0.45 () {56 vars in 1133 bytes} [Thu Aug 4 02:05:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:47.361581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753f852-ed84-458f-806d-0dfc7e1dc108 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:47.379693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671de22-1804-4a6d-91e6-d605883507cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:47.380314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671de22-1804-4a6d-91e6-d605883507cd None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:47.380596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671de22-1804-4a6d-91e6-d605883507cd None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:47.380897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671de22-1804-4a6d-91e6-d605883507cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:47.409834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:47.443126 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:47.450153 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671de22-1804-4a6d-91e6-d605883507cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:47.451608 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e671de22-1804-4a6d-91e6-d605883507cd None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:47.460423 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6753f852-ed84-458f-806d-0dfc7e1dc108 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:47.494062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86940d94-7d25-404d-8927-9c93e4393762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:47.494698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86940d94-7d25-404d-8927-9c93e4393762 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:47.494978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86940d94-7d25-404d-8927-9c93e4393762 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:47.495248 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86940d94-7d25-404d-8927-9c93e4393762 None admin] PATH_INFO: `/v3/projects/f2215af9cfee46539b78848b38d40863` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:47.496411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86940d94-7d25-404d-8927-9c93e4393762 None admin] RBAC: Authorizing `identity:get_project(project_id=f2215af9cfee46539b78848b38d40863)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:47.497877 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86940d94-7d25-404d-8927-9c93e4393762 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:47.499705 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 542/1075] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:05:47 2022] GET /identity/v3/projects/f2215af9cfee46539b78848b38d40863 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:47.500302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86940d94-7d25-404d-8927-9c93e4393762 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:47.555031 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 534/1076] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:05:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:05:47.556002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671de22-1804-4a6d-91e6-d605883507cd None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:47.608272 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86940d94-7d25-404d-8927-9c93e4393762 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:47.626648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:47.648009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c31aa-c85e-4bae-ba60-a1fa57e8c620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:47.648677 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c31aa-c85e-4bae-ba60-a1fa57e8c620 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:47.648962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c31aa-c85e-4bae-ba60-a1fa57e8c620 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:47.649237 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650c31aa-c85e-4bae-ba60-a1fa57e8c620 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:47.657886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:47.660246 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c31aa-c85e-4bae-ba60-a1fa57e8c620 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:47.661722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650c31aa-c85e-4bae-ba60-a1fa57e8c620 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:47.664959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671de22-1804-4a6d-91e6-d605883507cd None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:47.676209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-650c31aa-c85e-4bae-ba60-a1fa57e8c620 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:47.676209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:47.676209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:47.676209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:47.676209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:47.676209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:47.676209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:47.676209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:47.676209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:47.676209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:47.676209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:47.676209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:47.676209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:47.676209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:47.676209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:47.677334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:47.677334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:47.677334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:47.677334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:47.677334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:47.677334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:47.677334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:47.677334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:47.677334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:47.677334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:47.677334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:47.677334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:47.677334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:47.677334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:47.677334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:47.677334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:47.677334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:47.678437 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:47.678437 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:47.678437 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:47.678437 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:47.678437 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 543/1077] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:47.678437 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c31aa-c85e-4bae-ba60-a1fa57e8c620 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:47.702577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a54739-4397-434e-9c73-379f1b089b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:47.703161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a54739-4397-434e-9c73-379f1b089b47 None admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:47.703440 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a54739-4397-434e-9c73-379f1b089b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:47.703712 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a54739-4397-434e-9c73-379f1b089b47 None admin] PATH_INFO: `/v3/projects/f2215af9cfee46539b78848b38d40863/users/636376a57aad401fb3cd7ca83557bb86/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:47.706217 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a54739-4397-434e-9c73-379f1b089b47 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2215af9cfee46539b78848b38d40863, user_id=636376a57aad401fb3cd7ca83557bb86, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:47.707640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a54739-4397-434e-9c73-379f1b089b47 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:47.720585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 535/1078] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:05:47 2022] PUT /identity/v3/projects/f2215af9cfee46539b78848b38d40863/users/636376a57aad401fb3cd7ca83557bb86/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:05:47.721964 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a54739-4397-434e-9c73-379f1b089b47 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:47.791207 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650c31aa-c85e-4bae-ba60-a1fa57e8c620 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:47.832882 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487cb17-969c-4afa-86b4-da5ec6beeb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:47.833567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487cb17-969c-4afa-86b4-da5ec6beeb9c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:47.833845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487cb17-969c-4afa-86b4-da5ec6beeb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:47.834113 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487cb17-969c-4afa-86b4-da5ec6beeb9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:47.835383 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a54739-4397-434e-9c73-379f1b089b47 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:47.846521 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487cb17-969c-4afa-86b4-da5ec6beeb9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:47.848124 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487cb17-969c-4afa-86b4-da5ec6beeb9c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:47.864386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-7487cb17-969c-4afa-86b4-da5ec6beeb9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:47.864386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:47.864386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:47.864386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:47.864386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:47.864386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:47.864386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:47.864386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:47.864386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:47.864386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:47.864386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:47.864386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:47.864386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:47.864386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:47.864386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:47.865869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:47.865869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:47.865869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:47.865869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:47.865869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:47.865869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:47.865869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:47.865869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:47.865869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:47.865869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:47.865869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:47.865869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:47.865869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:47.865869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:47.865869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:47.865869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:47.865869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:47.866906 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:47.866906 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:47.866906 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:47.866906 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:47.866906 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 544/1079] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:47.873958 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487cb17-969c-4afa-86b4-da5ec6beeb9c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:47.883173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3712f6-732f-4db6-8d49-2e9b9bf050c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:47.883802 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3712f6-732f-4db6-8d49-2e9b9bf050c5 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:47.884087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3712f6-732f-4db6-8d49-2e9b9bf050c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:47.884361 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3712f6-732f-4db6-8d49-2e9b9bf050c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:47.885147 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3712f6-732f-4db6-8d49-2e9b9bf050c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:47.885453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3712f6-732f-4db6-8d49-2e9b9bf050c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:47.886840 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3712f6-732f-4db6-8d49-2e9b9bf050c5 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:47.900363 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 536/1080] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:47.905271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:47.906977 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3712f6-732f-4db6-8d49-2e9b9bf050c5 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:47.929842 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:47.996489 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487cb17-969c-4afa-86b4-da5ec6beeb9c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:48.029933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3712f6-732f-4db6-8d49-2e9b9bf050c5 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:48.035053 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97838a2-90cd-4a11-9422-62cabb7f28cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:48.035680 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97838a2-90cd-4a11-9422-62cabb7f28cb None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:48.035958 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97838a2-90cd-4a11-9422-62cabb7f28cb None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:48.036227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97838a2-90cd-4a11-9422-62cabb7f28cb None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:48.036949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97838a2-90cd-4a11-9422-62cabb7f28cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:48.037286 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97838a2-90cd-4a11-9422-62cabb7f28cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:48.038611 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97838a2-90cd-4a11-9422-62cabb7f28cb None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:48.050971 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 545/1081] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:48.068310 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c01811-8ac9-4d61-a384-030e9c97c917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:48.069003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c01811-8ac9-4d61-a384-030e9c97c917 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:48.069282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c01811-8ac9-4d61-a384-030e9c97c917 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:48.069554 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c01811-8ac9-4d61-a384-030e9c97c917 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:48.077402 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c01811-8ac9-4d61-a384-030e9c97c917 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:48.078886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c01811-8ac9-4d61-a384-030e9c97c917 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:48.093105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-83c01811-8ac9-4d61-a384-030e9c97c917 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:48.093105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:48.093105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:48.093105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:48.093105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:48.093105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:48.093105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:48.093105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:48.093105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:48.093105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:48.093105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:48.093105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:48.093105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:05:48.093105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:05:48.093105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:05:48.094096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:05:48.094096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:48.094096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:48.094096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:48.094096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:48.094096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:48.094096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:48.094096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:48.094096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:48.094096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:48.094096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:48.094096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:48.094096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:48.094096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:48.094096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:48.094096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:48.094096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:48.095211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:05:48.095211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:05:48.095211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:05:48.095211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:05:48.095211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:05:48.095211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:05:48.095211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:48.095211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:48.095211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 537/1082] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:48.100235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97838a2-90cd-4a11-9422-62cabb7f28cb None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:48.195666 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:48.222271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97838a2-90cd-4a11-9422-62cabb7f28cb None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:48.267891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dfc04a-b4ad-4788-bc28-46169ff128bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:48.268474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfc04a-b4ad-4788-bc28-46169ff128bc None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:48.268816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfc04a-b4ad-4788-bc28-46169ff128bc None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:48.269826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dfc04a-b4ad-4788-bc28-46169ff128bc None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:48.269826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfc04a-b4ad-4788-bc28-46169ff128bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:48.270131 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfc04a-b4ad-4788-bc28-46169ff128bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:48.271563 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dfc04a-b4ad-4788-bc28-46169ff128bc None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:48.287871 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 546/1083] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:05:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:48.297306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c01811-8ac9-4d61-a384-030e9c97c917 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:48.384087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:48.407040 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c01811-8ac9-4d61-a384-030e9c97c917 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:48.439676 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fcf2e3-eeec-4968-a342-0d9e3c339bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:48.440376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcf2e3-eeec-4968-a342-0d9e3c339bef None admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:48.440678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcf2e3-eeec-4968-a342-0d9e3c339bef None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:48.440951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcf2e3-eeec-4968-a342-0d9e3c339bef None admin] PATH_INFO: `/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/c571406d2a3f40dab035a918399d6f6a/roles/609fcaecc1a648f8b41c081fe0703ef4` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:48.443088 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcf2e3-eeec-4968-a342-0d9e3c339bef None admin] RBAC: Authorizing `identity:create_grant(project_id=74b8caff828a4f5289094fb2ae2278f7, user_id=c571406d2a3f40dab035a918399d6f6a, role_id=609fcaecc1a648f8b41c081fe0703ef4)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:48.444507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fcf2e3-eeec-4968-a342-0d9e3c339bef None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:48.457819 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 538/1084] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:05:48 2022] PUT /identity/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/c571406d2a3f40dab035a918399d6f6a/roles/609fcaecc1a648f8b41c081fe0703ef4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:05:49.780850 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b3eee-97cc-4090-b84a-1ff43dd07395 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:49.781414 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b3eee-97cc-4090-b84a-1ff43dd07395 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:49.781414 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b3eee-97cc-4090-b84a-1ff43dd07395 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:49.782789 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 547/1085] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:49.788701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b0178-6307-405c-b1e4-a3392e174f4b None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:49.789244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b0178-6307-405c-b1e4-a3392e174f4b None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:49.789444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b0178-6307-405c-b1e4-a3392e174f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:49.807720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-569b0178-6307-405c-b1e4-a3392e174f4b None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:49.808930 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-569b0178-6307-405c-b1e4-a3392e174f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:49.874228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 539/1086] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:49.897764 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b3eee-97cc-4090-b84a-1ff43dd07395 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:49.998339 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b3eee-97cc-4090-b84a-1ff43dd07395 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:50.033540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c4140-a23b-4869-88b5-0600a7eaa000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:50.034735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c4140-a23b-4869-88b5-0600a7eaa000 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:50.035019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c4140-a23b-4869-88b5-0600a7eaa000 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:50.035296 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c4140-a23b-4869-88b5-0600a7eaa000 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:50.043064 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:50.047343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c4140-a23b-4869-88b5-0600a7eaa000 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:05:50.053028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-de1c4140-a23b-4869-88b5-0600a7eaa000 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:05:50.053028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:50.053028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:50.053028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:50.053028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:50.053028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:50.053028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:50.053028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:50.053028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:50.053028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:50.053028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:50.053028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:50.053028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:05:50.053028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:05:50.053028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:05:50.053028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:05:50.059068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:05:50.059068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:05:50.059068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:05:50.059068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:05:50.059068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:50.059068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:50.059068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:50.059068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:50.059068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:50.059068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:50.059068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:50.059068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:50.059068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:50.059068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:50.059068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:50.059068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:50.060030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:50.060030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:50.060030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:50.060030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:50.060030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:05:50.060030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:05:50.060030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:05:50.060030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:05:50.060030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:05:50.060030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:05:50.060030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:05:50.060030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:50.060030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 548/1087] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:05:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:50.060781 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b0178-6307-405c-b1e4-a3392e174f4b None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:50.124406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:50.160145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b0178-6307-405c-b1e4-a3392e174f4b None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:50.193927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f24576-fc51-4eb1-bc57-cb8619a56adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:50.194541 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f24576-fc51-4eb1-bc57-cb8619a56adb None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:50.194874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f24576-fc51-4eb1-bc57-cb8619a56adb None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:50.195222 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f24576-fc51-4eb1-bc57-cb8619a56adb None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:50.195890 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f24576-fc51-4eb1-bc57-cb8619a56adb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:50.196194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f24576-fc51-4eb1-bc57-cb8619a56adb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:50.197587 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f24576-fc51-4eb1-bc57-cb8619a56adb None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:50.205595 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 540/1088] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:05:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:50.211876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c4140-a23b-4869-88b5-0600a7eaa000 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:50.312033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c4140-a23b-4869-88b5-0600a7eaa000 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:50.346835 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166aa8d6-661e-45c0-ac9d-df69cc54ac9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:50.347469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166aa8d6-661e-45c0-ac9d-df69cc54ac9c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:50.347749 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166aa8d6-661e-45c0-ac9d-df69cc54ac9c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:50.348022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166aa8d6-661e-45c0-ac9d-df69cc54ac9c None admin] PATH_INFO: `/v3/users/636376a57aad401fb3cd7ca83557bb86` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:50.349201 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166aa8d6-661e-45c0-ac9d-df69cc54ac9c None admin] RBAC: Authorizing `identity:get_user(user_id=636376a57aad401fb3cd7ca83557bb86)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:50.350622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166aa8d6-661e-45c0-ac9d-df69cc54ac9c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:50.352235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 549/1089] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:05:50 2022] GET /identity/v3/users/636376a57aad401fb3cd7ca83557bb86 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:50.358004 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f24576-fc51-4eb1-bc57-cb8619a56adb None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:50.455408 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f24576-fc51-4eb1-bc57-cb8619a56adb None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:50.459756 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fc17e-22fe-42c3-9c0e-506f66346bc3 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:50.460031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fc17e-22fe-42c3-9c0e-506f66346bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:50.460297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fc17e-22fe-42c3-9c0e-506f66346bc3 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:50.461691 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 550/1090] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:05:50.466614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97740c6-a1b6-44dc-8ec5-880b92e838d7 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:50.466886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97740c6-a1b6-44dc-8ec5-880b92e838d7 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:50.467160 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97740c6-a1b6-44dc-8ec5-880b92e838d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:50.481973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:50.487443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c97740c6-a1b6-44dc-8ec5-880b92e838d7 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:50.488636 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c97740c6-a1b6-44dc-8ec5-880b92e838d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:50.490743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12939192-30ca-42d6-93c7-2671f88b7421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:50.491400 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12939192-30ca-42d6-93c7-2671f88b7421 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:50.491645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12939192-30ca-42d6-93c7-2671f88b7421 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:50.491917 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12939192-30ca-42d6-93c7-2671f88b7421 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:50.549921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:50.556046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 551/1091] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 02:05:50.559962 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12939192-30ca-42d6-93c7-2671f88b7421 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:50.561477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12939192-30ca-42d6-93c7-2671f88b7421 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:50.580039 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97740c6-a1b6-44dc-8ec5-880b92e838d7 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:50.667053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 541/1092] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:05:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:05:50.672848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12939192-30ca-42d6-93c7-2671f88b7421 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:50.686825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97740c6-a1b6-44dc-8ec5-880b92e838d7 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:50.705442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:50.721851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e467ce8-6176-4bb7-9949-1f6f629e274e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:50.722495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e467ce8-6176-4bb7-9949-1f6f629e274e None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:50.722776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e467ce8-6176-4bb7-9949-1f6f629e274e None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:50.723052 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e467ce8-6176-4bb7-9949-1f6f629e274e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:50.733269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e467ce8-6176-4bb7-9949-1f6f629e274e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:05:50.738621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-3e467ce8-6176-4bb7-9949-1f6f629e274e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 02:05:50.738621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:50.738621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:50.738621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:50.738621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:50.738621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:50.738621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:50.738621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:50.738621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:50.738621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:50.738621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:50.738621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:50.738621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:05:50.738621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:05:50.738621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:05:50.738621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:05:50.739606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:05:50.739606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:05:50.739606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:05:50.739606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:05:50.739606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:50.739606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:50.739606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:50.739606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:50.739606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:50.739606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:50.739606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:50.739606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:50.739606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:50.739606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:50.739606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:50.739606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:50.740462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:50.740462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:50.740462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:50.740462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:50.740462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:05:50.740462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:05:50.740462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:05:50.740462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:05:50.740462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:05:50.740462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:05:50.740462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 04 02:05:50.740462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:50.740462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 552/1093] 10.222.0.45 () {56 vars in 1120 bytes} [Thu Aug 4 02:05:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:50.746193 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e467ce8-6176-4bb7-9949-1f6f629e274e None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:50.780253 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12939192-30ca-42d6-93c7-2671f88b7421 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:50.803168 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:50.814429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0b41c-6012-4177-8c92-c42fdd61e78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:50.815632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0b41c-6012-4177-8c92-c42fdd61e78c None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:50.815911 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0b41c-6012-4177-8c92-c42fdd61e78c None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:50.816209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db0b41c-6012-4177-8c92-c42fdd61e78c None admin] PATH_INFO: `/v3/projects/f2215af9cfee46539b78848b38d40863` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:50.817394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0b41c-6012-4177-8c92-c42fdd61e78c None admin] RBAC: Authorizing `identity:get_project(project_id=f2215af9cfee46539b78848b38d40863)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:50.818819 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db0b41c-6012-4177-8c92-c42fdd61e78c None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:50.820480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 542/1094] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:05:50 2022] GET /identity/v3/projects/f2215af9cfee46539b78848b38d40863 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:50.826521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0b41c-6012-4177-8c92-c42fdd61e78c None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:50.846019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e467ce8-6176-4bb7-9949-1f6f629e274e None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:50.880874 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c3711-da78-48e4-8dae-8fe441787faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:50.881484 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c3711-da78-48e4-8dae-8fe441787faf None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:50.881759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c3711-da78-48e4-8dae-8fe441787faf None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:50.882028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c3711-da78-48e4-8dae-8fe441787faf None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:50.882732 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c3711-da78-48e4-8dae-8fe441787faf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:50.883030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c3711-da78-48e4-8dae-8fe441787faf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:50.884379 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c3711-da78-48e4-8dae-8fe441787faf None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:50.892412 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 553/1095] 10.222.0.45 () {56 vars in 1115 bytes} [Thu Aug 4 02:05:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:05:50.901336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c3711-da78-48e4-8dae-8fe441787faf None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:50.920034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:50.927767 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db0b41c-6012-4177-8c92-c42fdd61e78c None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:50.961815 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced9e8a-c9c5-4239-8101-1b90d376206b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:50.962433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced9e8a-c9c5-4239-8101-1b90d376206b None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:50.962712 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced9e8a-c9c5-4239-8101-1b90d376206b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:50.963010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced9e8a-c9c5-4239-8101-1b90d376206b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:50.963847 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced9e8a-c9c5-4239-8101-1b90d376206b None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=f2215af9cfee46539b78848b38d40863, user.id=636376a57aad401fb3cd7ca83557bb86) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:50.964159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced9e8a-c9c5-4239-8101-1b90d376206b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:50.965608 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced9e8a-c9c5-4239-8101-1b90d376206b None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:50.971861 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 543/1096] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:05:50 2022] GET /identity/v3/role_assignments?user.id=636376a57aad401fb3cd7ca83557bb86&scope.project.id=f2215af9cfee46539b78848b38d40863&role.id=35205394c8af4c52a61b12f92ba16644 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:51.013440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c3711-da78-48e4-8dae-8fe441787faf None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:51.038547 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:51.055107 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e005a-fa72-44a2-8bc7-8d1598417bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:51.055767 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e005a-fa72-44a2-8bc7-8d1598417bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:51.056049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e005a-fa72-44a2-8bc7-8d1598417bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:51.056322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e005a-fa72-44a2-8bc7-8d1598417bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:51.068521 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e005a-fa72-44a2-8bc7-8d1598417bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:51.070053 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e005a-fa72-44a2-8bc7-8d1598417bf9 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:51.087077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-0a6e005a-fa72-44a2-8bc7-8d1598417bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:51.087077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:51.087077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:51.087077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:51.087077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:51.087077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:51.087077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:51.087077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:51.087077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:51.087077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:51.087077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:51.087077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:51.087077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:51.087077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:51.087077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:51.088072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:51.088072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:51.088072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:51.088072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:51.088072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:51.088072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:51.088072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:51.088072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:51.088072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:51.088072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:51.088072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:51.088072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:51.088072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:51.088072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:51.088072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:51.088072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:51.088072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:51.089194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:51.089194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:51.089194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:51.089194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:51.089194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 554/1097] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:51.094411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced9e8a-c9c5-4239-8101-1b90d376206b None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:51.197701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced9e8a-c9c5-4239-8101-1b90d376206b None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:51.231327 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64424029-d07a-47ef-a331-9fa609df061e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:51.231883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64424029-d07a-47ef-a331-9fa609df061e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:51.232164 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64424029-d07a-47ef-a331-9fa609df061e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:51.232429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64424029-d07a-47ef-a331-9fa609df061e None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:51.233136 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64424029-d07a-47ef-a331-9fa609df061e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:51.233435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64424029-d07a-47ef-a331-9fa609df061e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:51.234785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64424029-d07a-47ef-a331-9fa609df061e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:51.245795 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 544/1098] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:51.251382 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e005a-fa72-44a2-8bc7-8d1598417bf9 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:51.352547 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e005a-fa72-44a2-8bc7-8d1598417bf9 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:51.384945 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f977b-5a9d-413a-a47f-b5bfc31a0f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:51.385576 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f977b-5a9d-413a-a47f-b5bfc31a0f96 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:51.385853 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f977b-5a9d-413a-a47f-b5bfc31a0f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:51.386116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f977b-5a9d-413a-a47f-b5bfc31a0f96 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:51.393949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f977b-5a9d-413a-a47f-b5bfc31a0f96 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:51.395363 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f977b-5a9d-413a-a47f-b5bfc31a0f96 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:51.406340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-194f977b-5a9d-413a-a47f-b5bfc31a0f96 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:05:51.406340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:51.406340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:51.406340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:51.406340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:51.406340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:51.406340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:51.406340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:51.406340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:51.406340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:51.406340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:51.406340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:51.406340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:05:51.406340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:05:51.406340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:05:51.406340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:51.407490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:51.408466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:51.408466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:51.408466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:05:51.408466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:05:51.408466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:05:51.408466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:05:51.408466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:05:51.408466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:05:51.408466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 04 02:05:51.408466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:05:51.408466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 555/1099] 10.222.0.45 () {56 vars in 1093 bytes} [Thu Aug 4 02:05:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:05:51.413703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64424029-d07a-47ef-a331-9fa609df061e None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:51.435259 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:51.511872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64424029-d07a-47ef-a331-9fa609df061e None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:51.545106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf9e70-30d2-407b-9499-733f1de20521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:51.545717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf9e70-30d2-407b-9499-733f1de20521 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:51.545997 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf9e70-30d2-407b-9499-733f1de20521 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:51.546273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaf9e70-30d2-407b-9499-733f1de20521 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:51.547057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf9e70-30d2-407b-9499-733f1de20521 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:51.547364 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf9e70-30d2-407b-9499-733f1de20521 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:51.548820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaf9e70-30d2-407b-9499-733f1de20521 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:51.570823 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 545/1100] 10.222.0.45 () {56 vars in 1133 bytes} [Thu Aug 4 02:05:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:51.572175 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f977b-5a9d-413a-a47f-b5bfc31a0f96 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:51.603567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:51.674192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f977b-5a9d-413a-a47f-b5bfc31a0f96 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:51.708823 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade92c51-45ee-4c3c-81c4-d42c27ed0a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:51.709443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade92c51-45ee-4c3c-81c4-d42c27ed0a6f None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:51.709724 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade92c51-45ee-4c3c-81c4-d42c27ed0a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:51.710013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade92c51-45ee-4c3c-81c4-d42c27ed0a6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:51.777052 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade92c51-45ee-4c3c-81c4-d42c27ed0a6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:51.778413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade92c51-45ee-4c3c-81c4-d42c27ed0a6f None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:51.807283 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:51.874564 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 556/1101] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:05:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:05:51.880895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf9e70-30d2-407b-9499-733f1de20521 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:51.958260 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:51.979317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaf9e70-30d2-407b-9499-733f1de20521 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:52.012516 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27333d12-b42d-4731-b99f-a304e8689ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:52.013165 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27333d12-b42d-4731-b99f-a304e8689ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:52.013441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27333d12-b42d-4731-b99f-a304e8689ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:52.013710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27333d12-b42d-4731-b99f-a304e8689ab7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:52.023402 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27333d12-b42d-4731-b99f-a304e8689ab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:52.024870 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27333d12-b42d-4731-b99f-a304e8689ab7 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:52.038899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-27333d12-b42d-4731-b99f-a304e8689ab7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:52.038899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:52.038899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:52.038899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:52.038899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:52.038899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:52.038899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:52.038899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:52.038899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:52.038899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:52.038899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:52.038899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:52.038899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:05:52.038899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:05:52.038899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:05:52.039886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:05:52.039886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:52.039886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:52.039886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:52.039886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:52.039886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:52.039886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:52.039886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:52.039886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:52.039886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:52.039886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:52.039886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:52.039886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:52.039886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:52.039886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:52.039886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:52.039886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:52.041227 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:05:52.041227 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:05:52.041227 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:05:52.041227 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:52.041227 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 546/1102] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:52.045686 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade92c51-45ee-4c3c-81c4-d42c27ed0a6f None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:52.146846 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade92c51-45ee-4c3c-81c4-d42c27ed0a6f None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:52.175737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:52.181168 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c02a85-00e7-4faf-b2d3-62d2cfa86433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:52.181778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c02a85-00e7-4faf-b2d3-62d2cfa86433 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:52.182056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c02a85-00e7-4faf-b2d3-62d2cfa86433 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:52.182327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c02a85-00e7-4faf-b2d3-62d2cfa86433 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:52.183013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c02a85-00e7-4faf-b2d3-62d2cfa86433 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:52.183317 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c02a85-00e7-4faf-b2d3-62d2cfa86433 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:52.184701 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c02a85-00e7-4faf-b2d3-62d2cfa86433 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:52.196726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 557/1103] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:52.202343 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27333d12-b42d-4731-b99f-a304e8689ab7 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:52.298659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27333d12-b42d-4731-b99f-a304e8689ab7 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:52.326831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:52.333403 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13e6f0-2f34-4c05-9620-9ab01ca97b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:52.334041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13e6f0-2f34-4c05-9620-9ab01ca97b16 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:52.334316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13e6f0-2f34-4c05-9620-9ab01ca97b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:52.334585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13e6f0-2f34-4c05-9620-9ab01ca97b16 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:52.341119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13e6f0-2f34-4c05-9620-9ab01ca97b16 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:52.342574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13e6f0-2f34-4c05-9620-9ab01ca97b16 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:52.353898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-6b13e6f0-2f34-4c05-9620-9ab01ca97b16 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:52.353898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:05:52.353898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:05:52.353898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:05:52.353898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:05:52.353898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:05:52.353898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:05:52.353898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:05:52.353898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:05:52.353898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:05:52.353898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:05:52.353898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:05:52.353898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:05:52.353898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:05:52.353898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:05:52.354901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:05:52.354901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:05:52.354901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:05:52.354901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:05:52.354901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:05:52.354901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:05:52.354901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:05:52.354901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:05:52.354901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:05:52.354901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:05:52.354901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:05:52.354901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:05:52.354901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:05:52.354901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:05:52.354901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:05:52.354901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:05:52.354901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:05:52.355800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:05:52.355800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:05:52.355800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:05:52.355800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:05:52.355800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:05:52.355800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:05:52.355800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:05:52.355800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:05:52.355800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 547/1104] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:05:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:05:52.360934 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c02a85-00e7-4faf-b2d3-62d2cfa86433 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:52.458545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c02a85-00e7-4faf-b2d3-62d2cfa86433 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:52.498314 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a4eb6-efe3-4a86-b1d1-82970a8b1214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:52.498927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a4eb6-efe3-4a86-b1d1-82970a8b1214 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:52.499207 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a4eb6-efe3-4a86-b1d1-82970a8b1214 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:52.499481 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a4eb6-efe3-4a86-b1d1-82970a8b1214 None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:52.500231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a4eb6-efe3-4a86-b1d1-82970a8b1214 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:52.500546 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a4eb6-efe3-4a86-b1d1-82970a8b1214 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:52.501999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a4eb6-efe3-4a86-b1d1-82970a8b1214 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:52.514474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 558/1105] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:05:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:52.521301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13e6f0-2f34-4c05-9620-9ab01ca97b16 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:52.622184 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13e6f0-2f34-4c05-9620-9ab01ca97b16 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:52.655970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffabe1a6-45c2-4d6d-aaf3-4cc4c381c0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:52.656590 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabe1a6-45c2-4d6d-aaf3-4cc4c381c0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:52.656933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabe1a6-45c2-4d6d-aaf3-4cc4c381c0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:52.657144 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffabe1a6-45c2-4d6d-aaf3-4cc4c381c0c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:52.657928 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabe1a6-45c2-4d6d-aaf3-4cc4c381c0c1 None admin] RBAC: Adding query filter params (role.id=609fcaecc1a648f8b41c081fe0703ef4, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=c571406d2a3f40dab035a918399d6f6a) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:52.658316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabe1a6-45c2-4d6d-aaf3-4cc4c381c0c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:52.659685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffabe1a6-45c2-4d6d-aaf3-4cc4c381c0c1 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:52.665649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 548/1106] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:05:52 2022] GET /identity/v3/role_assignments?user.id=c571406d2a3f40dab035a918399d6f6a&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=609fcaecc1a648f8b41c081fe0703ef4 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:53.017199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51740d77-afc2-4390-a948-31a2d2297aa0 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:53.017698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51740d77-afc2-4390-a948-31a2d2297aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:53.017698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51740d77-afc2-4390-a948-31a2d2297aa0 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:53.019506 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 559/1107] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:53.024906 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bacc13-c8fe-4cf9-800a-7486e1364a72 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:53.025352 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bacc13-c8fe-4cf9-800a-7486e1364a72 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:53.025438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bacc13-c8fe-4cf9-800a-7486e1364a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:53.043857 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-d0bacc13-c8fe-4cf9-800a-7486e1364a72 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:53.045070 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d0bacc13-c8fe-4cf9-800a-7486e1364a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:53.096048 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:53.115658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 549/1108] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:53.141281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51740d77-afc2-4390-a948-31a2d2297aa0 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:53.199674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:53.250009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51740d77-afc2-4390-a948-31a2d2297aa0 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:53.284915 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b5bdb-552b-4da1-ac9e-eb0fab0563ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:53.285483 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b5bdb-552b-4da1-ac9e-eb0fab0563ee None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:53.285814 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b5bdb-552b-4da1-ac9e-eb0fab0563ee None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:53.286090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b5bdb-552b-4da1-ac9e-eb0fab0563ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:53.287859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b5bdb-552b-4da1-ac9e-eb0fab0563ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:53.289314 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b5bdb-552b-4da1-ac9e-eb0fab0563ee None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:53.290946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 560/1109] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:05:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:53.297216 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bacc13-c8fe-4cf9-800a-7486e1364a72 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:53.404159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bacc13-c8fe-4cf9-800a-7486e1364a72 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:53.437391 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af6065-77f5-4322-b8af-29b95d3f4d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:53.438007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af6065-77f5-4322-b8af-29b95d3f4d79 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:53.438279 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af6065-77f5-4322-b8af-29b95d3f4d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:53.438546 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af6065-77f5-4322-b8af-29b95d3f4d79 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:53.439489 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af6065-77f5-4322-b8af-29b95d3f4d79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:53.441439 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11af6065-77f5-4322-b8af-29b95d3f4d79 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:53.464438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 550/1110] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:05:53 2022] POST /identity/v3/users => generated 286 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:05:55.488519 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3cb8a-e9fd-4c23-a3d9-713d5902367a None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:55.489103 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3cb8a-e9fd-4c23-a3d9-713d5902367a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:55.489103 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d3cb8a-e9fd-4c23-a3d9-713d5902367a None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:55.490416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 561/1111] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:55.495540 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a58753-1f75-4656-83fb-9afd6d5c3205 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:55.495947 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a58753-1f75-4656-83fb-9afd6d5c3205 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:55.496079 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a58753-1f75-4656-83fb-9afd6d5c3205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:55.513622 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-a1a58753-1f75-4656-83fb-9afd6d5c3205 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:55.514799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a1a58753-1f75-4656-83fb-9afd6d5c3205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:55.547541 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:55.580718 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 551/1112] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:55.604210 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3cb8a-e9fd-4c23-a3d9-713d5902367a None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:55.709904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d3cb8a-e9fd-4c23-a3d9-713d5902367a None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:55.729912 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:55.744055 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a754a1-a179-4cfe-b4d5-92c8406e241c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:55.744651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a754a1-a179-4cfe-b4d5-92c8406e241c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:55.744933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a754a1-a179-4cfe-b4d5-92c8406e241c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:55.745209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a754a1-a179-4cfe-b4d5-92c8406e241c None admin] PATH_INFO: `/v3/roles/80e5528d7c8742f78e29e64c1216176c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:55.746655 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a754a1-a179-4cfe-b4d5-92c8406e241c None admin] RBAC: Authorizing `identity:get_role(role_id=80e5528d7c8742f78e29e64c1216176c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:55.748024 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a754a1-a179-4cfe-b4d5-92c8406e241c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:55.749412 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 562/1113] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:05:55 2022] GET /identity/v3/roles/80e5528d7c8742f78e29e64c1216176c => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:55.755332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a58753-1f75-4656-83fb-9afd6d5c3205 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:55.850220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a58753-1f75-4656-83fb-9afd6d5c3205 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:55.881693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb0203-4a30-427d-a129-8b48c5b02655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:55.882314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb0203-4a30-427d-a129-8b48c5b02655 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:55.882588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb0203-4a30-427d-a129-8b48c5b02655 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:55.882857 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb0203-4a30-427d-a129-8b48c5b02655 None admin] PATH_INFO: `/v3/users/36f4ace2ca0943cbaf517621c082823d` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:55.895941 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb0203-4a30-427d-a129-8b48c5b02655 None admin] RBAC: Authorizing `identity:get_user(user_id=36f4ace2ca0943cbaf517621c082823d)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:55.897415 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb0203-4a30-427d-a129-8b48c5b02655 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:55.899074 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 552/1114] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:05:55 2022] GET /identity/v3/users/36f4ace2ca0943cbaf517621c082823d => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:55.904925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a754a1-a179-4cfe-b4d5-92c8406e241c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:56.001630 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a754a1-a179-4cfe-b4d5-92c8406e241c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:56.035072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1acc138-bde0-465d-ba4b-9a755100d79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:56.035678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acc138-bde0-465d-ba4b-9a755100d79c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:56.035961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acc138-bde0-465d-ba4b-9a755100d79c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:56.036236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1acc138-bde0-465d-ba4b-9a755100d79c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:56.090691 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:56.103386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1acc138-bde0-465d-ba4b-9a755100d79c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:56.104853 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1acc138-bde0-465d-ba4b-9a755100d79c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:56.200483 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 563/1115] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:05:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:05:56.207022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb0203-4a30-427d-a129-8b48c5b02655 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:56.233810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:56.306920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb0203-4a30-427d-a129-8b48c5b02655 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:56.342682 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17377a6-450c-4a13-a2d0-cde9c380a485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:56.343314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17377a6-450c-4a13-a2d0-cde9c380a485 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:56.343594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17377a6-450c-4a13-a2d0-cde9c380a485 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:56.343867 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17377a6-450c-4a13-a2d0-cde9c380a485 None admin] PATH_INFO: `/v3/projects/f2215af9cfee46539b78848b38d40863` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:56.345047 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17377a6-450c-4a13-a2d0-cde9c380a485 None admin] RBAC: Authorizing `identity:get_project(project_id=f2215af9cfee46539b78848b38d40863)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:56.346505 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17377a6-450c-4a13-a2d0-cde9c380a485 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:56.348200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 553/1116] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:05:56 2022] GET /identity/v3/projects/f2215af9cfee46539b78848b38d40863 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:56.354331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1acc138-bde0-465d-ba4b-9a755100d79c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:56.437948 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:56.450991 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1acc138-bde0-465d-ba4b-9a755100d79c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:56.484211 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b783c863-bbd5-4a8c-899f-0272797933b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:56.484840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783c863-bbd5-4a8c-899f-0272797933b3 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:56.485121 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783c863-bbd5-4a8c-899f-0272797933b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:56.485393 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b783c863-bbd5-4a8c-899f-0272797933b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:56.486168 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783c863-bbd5-4a8c-899f-0272797933b3 None admin] RBAC: Adding query filter params (role.id=80e5528d7c8742f78e29e64c1216176c, scope.project.id=f2215af9cfee46539b78848b38d40863, user.id=36f4ace2ca0943cbaf517621c082823d) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:05:56.486475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783c863-bbd5-4a8c-899f-0272797933b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:56.488409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b783c863-bbd5-4a8c-899f-0272797933b3 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:56.493676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 564/1117] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:05:56 2022] GET /identity/v3/role_assignments?user.id=36f4ace2ca0943cbaf517621c082823d&scope.project.id=f2215af9cfee46539b78848b38d40863&role.id=80e5528d7c8742f78e29e64c1216176c => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:58.464271 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5b139-0caa-4927-ba68-eb70562db997 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:58.464874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5b139-0caa-4927-ba68-eb70562db997 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:58.465068 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5b139-0caa-4927-ba68-eb70562db997 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:58.466654 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 554/1118] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:05:58.472580 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d717ca-b447-4649-af57-6c09c62d1ad7 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:58.473025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d717ca-b447-4649-af57-6c09c62d1ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:58.473147 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d717ca-b447-4649-af57-6c09c62d1ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:58.491648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-34d717ca-b447-4649-af57-6c09c62d1ad7 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:05:58.492904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-34d717ca-b447-4649-af57-6c09c62d1ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:05:58.558299 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 565/1119] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:05:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:05:58.582161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5b139-0caa-4927-ba68-eb70562db997 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:58.674125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:58.681699 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5b139-0caa-4927-ba68-eb70562db997 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:58.714911 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75348792-8cf6-43a8-bff0-93bb8a5145a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:58.715478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75348792-8cf6-43a8-bff0-93bb8a5145a8 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:58.715759 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75348792-8cf6-43a8-bff0-93bb8a5145a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:58.716033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75348792-8cf6-43a8-bff0-93bb8a5145a8 None admin] PATH_INFO: `/v3/roles/80e5528d7c8742f78e29e64c1216176c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:58.717510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75348792-8cf6-43a8-bff0-93bb8a5145a8 None admin] RBAC: Authorizing `identity:get_role(role_id=80e5528d7c8742f78e29e64c1216176c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:58.718879 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75348792-8cf6-43a8-bff0-93bb8a5145a8 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:58.720219 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 555/1120] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:05:58 2022] GET /identity/v3/roles/80e5528d7c8742f78e29e64c1216176c => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:58.726679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d717ca-b447-4649-af57-6c09c62d1ad7 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:58.804869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:05:58.823579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d717ca-b447-4649-af57-6c09c62d1ad7 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:58.855682 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247a7f1-e809-4106-bde6-22c7d37be41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:58.856307 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247a7f1-e809-4106-bde6-22c7d37be41c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:58.856587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247a7f1-e809-4106-bde6-22c7d37be41c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:58.856892 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247a7f1-e809-4106-bde6-22c7d37be41c None admin] PATH_INFO: `/v3/users/36f4ace2ca0943cbaf517621c082823d` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:58.858044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247a7f1-e809-4106-bde6-22c7d37be41c None admin] RBAC: Authorizing `identity:get_user(user_id=36f4ace2ca0943cbaf517621c082823d)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:58.859459 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247a7f1-e809-4106-bde6-22c7d37be41c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:58.861099 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 566/1121] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:05:58 2022] GET /identity/v3/users/36f4ace2ca0943cbaf517621c082823d => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:58.867370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75348792-8cf6-43a8-bff0-93bb8a5145a8 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:58.965949 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75348792-8cf6-43a8-bff0-93bb8a5145a8 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:58.999432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae695c6-01ab-4729-b7ce-56638c2c346e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:59.000053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae695c6-01ab-4729-b7ce-56638c2c346e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:59.000333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae695c6-01ab-4729-b7ce-56638c2c346e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:59.000607 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae695c6-01ab-4729-b7ce-56638c2c346e None admin] PATH_INFO: `/v3/projects/f2215af9cfee46539b78848b38d40863` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:59.001789 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae695c6-01ab-4729-b7ce-56638c2c346e None admin] RBAC: Authorizing `identity:get_project(project_id=f2215af9cfee46539b78848b38d40863)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:59.003773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae695c6-01ab-4729-b7ce-56638c2c346e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:59.005460 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 556/1122] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:05:58 2022] GET /identity/v3/projects/f2215af9cfee46539b78848b38d40863 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:05:59.011541 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247a7f1-e809-4106-bde6-22c7d37be41c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:05:59.111168 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247a7f1-e809-4106-bde6-22c7d37be41c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:05:59.145304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0dd25-7147-4a7b-abc2-fad1283efad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:05:59.145961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0dd25-7147-4a7b-abc2-fad1283efad2 None admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:05:59.146243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0dd25-7147-4a7b-abc2-fad1283efad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:05:59.146521 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0dd25-7147-4a7b-abc2-fad1283efad2 None admin] PATH_INFO: `/v3/projects/f2215af9cfee46539b78848b38d40863/users/36f4ace2ca0943cbaf517621c082823d/roles/80e5528d7c8742f78e29e64c1216176c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:05:59.148697 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0dd25-7147-4a7b-abc2-fad1283efad2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2215af9cfee46539b78848b38d40863, user_id=36f4ace2ca0943cbaf517621c082823d, role_id=80e5528d7c8742f78e29e64c1216176c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:05:59.150114 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d0dd25-7147-4a7b-abc2-fad1283efad2 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:05:59.163247 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 567/1123] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:05:59 2022] PUT /identity/v3/projects/f2215af9cfee46539b78848b38d40863/users/36f4ace2ca0943cbaf517621c082823d/roles/80e5528d7c8742f78e29e64c1216176c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:06:01.160409 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1cef8-a3e8-4e07-89e2-21ce4a196c50 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:01.161024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1cef8-a3e8-4e07-89e2-21ce4a196c50 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:01.161024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde1cef8-a3e8-4e07-89e2-21ce4a196c50 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:01.162335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 557/1124] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:06:01.167552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed71c0-a665-42cf-a6ad-c0132e0fe7fe None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:01.167982 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed71c0-a665-42cf-a6ad-c0132e0fe7fe None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:01.168091 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed71c0-a665-42cf-a6ad-c0132e0fe7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:01.186073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-d5ed71c0-a665-42cf-a6ad-c0132e0fe7fe None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:06:01.187256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d5ed71c0-a665-42cf-a6ad-c0132e0fe7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:06:01.190911 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:01.252261 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 568/1125] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:06:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 04 02:06:01.275417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1cef8-a3e8-4e07-89e2-21ce4a196c50 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:01.305400 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:01.374401 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde1cef8-a3e8-4e07-89e2-21ce4a196c50 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:01.408312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2375e6-eca7-40ff-8990-f02362a46ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:01.409486 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2375e6-eca7-40ff-8990-f02362a46ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:01.409761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2375e6-eca7-40ff-8990-f02362a46ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:01.410029 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2375e6-eca7-40ff-8990-f02362a46ff8 None admin] PATH_INFO: `/v3/roles/80e5528d7c8742f78e29e64c1216176c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:01.411507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2375e6-eca7-40ff-8990-f02362a46ff8 None admin] RBAC: Authorizing `identity:get_role(role_id=80e5528d7c8742f78e29e64c1216176c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:01.412845 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2375e6-eca7-40ff-8990-f02362a46ff8 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:01.414461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 558/1126] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:06:01 2022] GET /identity/v3/roles/80e5528d7c8742f78e29e64c1216176c => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:01.420033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed71c0-a665-42cf-a6ad-c0132e0fe7fe None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:01.527795 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed71c0-a665-42cf-a6ad-c0132e0fe7fe None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:01.564372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8625fb8-e30e-42f3-9b93-0334a8c7112b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:01.565039 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8625fb8-e30e-42f3-9b93-0334a8c7112b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:01.565313 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8625fb8-e30e-42f3-9b93-0334a8c7112b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:01.565584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8625fb8-e30e-42f3-9b93-0334a8c7112b None admin] PATH_INFO: `/v3/users/36f4ace2ca0943cbaf517621c082823d` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:01.566744 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8625fb8-e30e-42f3-9b93-0334a8c7112b None admin] RBAC: Authorizing `identity:get_user(user_id=36f4ace2ca0943cbaf517621c082823d)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:01.568164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8625fb8-e30e-42f3-9b93-0334a8c7112b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:01.569802 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 569/1127] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:06:01 2022] GET /identity/v3/users/36f4ace2ca0943cbaf517621c082823d => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:06:01.575682 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2375e6-eca7-40ff-8990-f02362a46ff8 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:01.671892 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:01.675526 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2375e6-eca7-40ff-8990-f02362a46ff8 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:01.709093 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8621472-d637-436e-b323-afd23081266e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:01.709696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8621472-d637-436e-b323-afd23081266e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:01.709975 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8621472-d637-436e-b323-afd23081266e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:01.710245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8621472-d637-436e-b323-afd23081266e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:01.774471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8621472-d637-436e-b323-afd23081266e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:01.775913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8621472-d637-436e-b323-afd23081266e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:01.866583 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:01.870526 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 559/1128] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:06:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:06:01.876522 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8625fb8-e30e-42f3-9b93-0334a8c7112b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:01.887143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:01.973582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8625fb8-e30e-42f3-9b93-0334a8c7112b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:02.005343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b547808f-ec05-4f55-8e89-30088d4ad9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:02.005970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547808f-ec05-4f55-8e89-30088d4ad9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:02.006249 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547808f-ec05-4f55-8e89-30088d4ad9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:02.006520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b547808f-ec05-4f55-8e89-30088d4ad9a0 None admin] PATH_INFO: `/v3/projects/f2215af9cfee46539b78848b38d40863` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:02.007755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547808f-ec05-4f55-8e89-30088d4ad9a0 None admin] RBAC: Authorizing `identity:get_project(project_id=f2215af9cfee46539b78848b38d40863)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:02.009201 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b547808f-ec05-4f55-8e89-30088d4ad9a0 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:02.010852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 570/1129] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:06:01 2022] GET /identity/v3/projects/f2215af9cfee46539b78848b38d40863 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:02.017030 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8621472-d637-436e-b323-afd23081266e None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:02.112988 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8621472-d637-436e-b323-afd23081266e None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:02.146080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed9d95-3cc4-49c9-a7f5-7f21ea50b757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:02.146687 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed9d95-3cc4-49c9-a7f5-7f21ea50b757 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:02.146968 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed9d95-3cc4-49c9-a7f5-7f21ea50b757 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:02.147240 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed9d95-3cc4-49c9-a7f5-7f21ea50b757 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:02.148012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed9d95-3cc4-49c9-a7f5-7f21ea50b757 None admin] RBAC: Adding query filter params (role.id=80e5528d7c8742f78e29e64c1216176c, scope.project.id=f2215af9cfee46539b78848b38d40863, user.id=36f4ace2ca0943cbaf517621c082823d) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:06:02.148317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed9d95-3cc4-49c9-a7f5-7f21ea50b757 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:02.149755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed9d95-3cc4-49c9-a7f5-7f21ea50b757 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:02.155363 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 560/1130] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:06:02 2022] GET /identity/v3/role_assignments?user.id=36f4ace2ca0943cbaf517621c082823d&scope.project.id=f2215af9cfee46539b78848b38d40863&role.id=80e5528d7c8742f78e29e64c1216176c => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:04.198995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87ffb2-35f3-4154-8bbf-ce7b7f194d5a None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:04.199550 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87ffb2-35f3-4154-8bbf-ce7b7f194d5a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:04.199550 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87ffb2-35f3-4154-8bbf-ce7b7f194d5a None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:04.200921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 571/1131] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:06:04.206047 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e3226-6a2d-4b2c-9091-e8967bf3bdf7 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:04.206421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e3226-6a2d-4b2c-9091-e8967bf3bdf7 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:04.206588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e3226-6a2d-4b2c-9091-e8967bf3bdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:04.224283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-841e3226-6a2d-4b2c-9091-e8967bf3bdf7 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:06:04.225488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-841e3226-6a2d-4b2c-9091-e8967bf3bdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:06:04.282418 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:04.291365 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 561/1132] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:06:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:06:04.315682 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87ffb2-35f3-4154-8bbf-ce7b7f194d5a None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:04.391533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:04.417212 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a87ffb2-35f3-4154-8bbf-ce7b7f194d5a None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:04.451864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5788d73f-308d-42b7-be8b-84ef1f4a2568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:04.452417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788d73f-308d-42b7-be8b-84ef1f4a2568 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:04.452722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788d73f-308d-42b7-be8b-84ef1f4a2568 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:04.452993 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5788d73f-308d-42b7-be8b-84ef1f4a2568 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:04.454152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788d73f-308d-42b7-be8b-84ef1f4a2568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:04.455572 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5788d73f-308d-42b7-be8b-84ef1f4a2568 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:04.457173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 572/1133] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:06:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:04.463156 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e3226-6a2d-4b2c-9091-e8967bf3bdf7 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:04.558669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e3226-6a2d-4b2c-9091-e8967bf3bdf7 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:04.591068 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb723d2c-cb02-4071-858a-0eafb05a0721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:04.591675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb723d2c-cb02-4071-858a-0eafb05a0721 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:04.591951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb723d2c-cb02-4071-858a-0eafb05a0721 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:04.592219 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb723d2c-cb02-4071-858a-0eafb05a0721 None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:04.592915 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb723d2c-cb02-4071-858a-0eafb05a0721 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:04.594341 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb723d2c-cb02-4071-858a-0eafb05a0721 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:04.621636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 562/1134] 10.222.0.45 () {60 vars in 1141 bytes} [Thu Aug 4 02:06:04 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:06:06.675368 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1346ad-1fa2-46e3-a86e-15eeb5dc297d None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:06.675801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1346ad-1fa2-46e3-a86e-15eeb5dc297d None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:06.675906 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1346ad-1fa2-46e3-a86e-15eeb5dc297d None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:06.677307 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 573/1135] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:06:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:06:06.682570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d5614-0375-4996-93a5-406724ebc04d None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:06.682943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d5614-0375-4996-93a5-406724ebc04d None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:06.683109 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d5614-0375-4996-93a5-406724ebc04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:06.701098 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-2e2d5614-0375-4996-93a5-406724ebc04d None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:06:06.702279 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2e2d5614-0375-4996-93a5-406724ebc04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:06:06.743041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:06.767591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 563/1136] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:06:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:06:06.792180 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1346ad-1fa2-46e3-a86e-15eeb5dc297d None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:06.889650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1346ad-1fa2-46e3-a86e-15eeb5dc297d None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:06.925755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c31b1-d9a8-4ebd-9031-496513fe534d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:06.926308 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c31b1-d9a8-4ebd-9031-496513fe534d None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:06.926584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c31b1-d9a8-4ebd-9031-496513fe534d None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:06.926853 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c31b1-d9a8-4ebd-9031-496513fe534d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:06.927999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c31b1-d9a8-4ebd-9031-496513fe534d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:06.929498 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c31b1-d9a8-4ebd-9031-496513fe534d None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:06.931095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 574/1137] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:06:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:06:06.936914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d5614-0375-4996-93a5-406724ebc04d None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:07.033197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d5614-0375-4996-93a5-406724ebc04d None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:07.066256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5801d5c-62fb-4290-a294-0978f9173873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:07.066865 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5801d5c-62fb-4290-a294-0978f9173873 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:07.067146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5801d5c-62fb-4290-a294-0978f9173873 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:07.067421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5801d5c-62fb-4290-a294-0978f9173873 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:07.068109 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5801d5c-62fb-4290-a294-0978f9173873 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:07.069560 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5801d5c-62fb-4290-a294-0978f9173873 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:07.090961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 564/1138] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:06:06 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:06:09.139012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7c7a1-3417-4a4d-ac3d-ea4c6e87c655 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:09.139555 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7c7a1-3417-4a4d-ac3d-ea4c6e87c655 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:09.139555 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7c7a1-3417-4a4d-ac3d-ea4c6e87c655 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:09.140943 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 575/1139] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:06:09.146763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9ab15-d377-4a3e-bb16-30fd89eae012 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:09.147146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9ab15-d377-4a3e-bb16-30fd89eae012 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:09.147307 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9ab15-d377-4a3e-bb16-30fd89eae012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:09.165344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-d7b9ab15-d377-4a3e-bb16-30fd89eae012 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:06:09.166527 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d7b9ab15-d377-4a3e-bb16-30fd89eae012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:06:09.192099 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:09.233624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 565/1140] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:06:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:06:09.256767 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7c7a1-3417-4a4d-ac3d-ea4c6e87c655 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:09.271646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:09.356346 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c7c7a1-3417-4a4d-ac3d-ea4c6e87c655 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:09.388232 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e440b-31d4-4f69-870f-49cf290b3bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:09.388877 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e440b-31d4-4f69-870f-49cf290b3bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:09.389208 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e440b-31d4-4f69-870f-49cf290b3bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:09.389424 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7e440b-31d4-4f69-870f-49cf290b3bd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:09.399587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7e440b-31d4-4f69-870f-49cf290b3bd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:06:09.404323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-2b7e440b-31d4-4f69-870f-49cf290b3bd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:06:09.404323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:06:09.404323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:06:09.404323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:06:09.404323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:06:09.404323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:06:09.404323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:06:09.404323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:06:09.404323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:06:09.404323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:06:09.404323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:06:09.404323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:06:09.404323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:06:09.404323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:06:09.404323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:06:09.404323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:06:09.405544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:06:09.405544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:06:09.405544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:06:09.405544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:06:09.405544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:06:09.405544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:06:09.405544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:06:09.405544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:06:09.405544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:06:09.405544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:06:09.405544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:06:09.405544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:06:09.405544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:06:09.405544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:06:09.405544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:06:09.405544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:06:09.406609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:06:09.406609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:06:09.406609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:06:09.406609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:06:09.406609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:06:09.406609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:06:09.406609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:06:09.406609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:06:09.406609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:06:09.406609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:06:09.406609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:06:09.406609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:06:09.406609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 576/1141] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:06:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:06:09.412214 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9ab15-d377-4a3e-bb16-30fd89eae012 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:09.507296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9ab15-d377-4a3e-bb16-30fd89eae012 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:09.540376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0207ec-d908-4714-bef7-7c21e3e083cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:09.541008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0207ec-d908-4714-bef7-7c21e3e083cb None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:09.541282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0207ec-d908-4714-bef7-7c21e3e083cb None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:09.541551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0207ec-d908-4714-bef7-7c21e3e083cb None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:09.542259 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0207ec-d908-4714-bef7-7c21e3e083cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:06:09.542565 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0207ec-d908-4714-bef7-7c21e3e083cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:09.543919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0207ec-d908-4714-bef7-7c21e3e083cb None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:09.551735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 566/1142] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:06:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:09.557580 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e440b-31d4-4f69-870f-49cf290b3bd4 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:09.626681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:09.656709 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7e440b-31d4-4f69-870f-49cf290b3bd4 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:09.691054 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e785b3-a225-4777-b9a7-9f2c37d53e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:09.691694 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e785b3-a225-4777-b9a7-9f2c37d53e7a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:09.692014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e785b3-a225-4777-b9a7-9f2c37d53e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:09.692241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e785b3-a225-4777-b9a7-9f2c37d53e7a None admin] PATH_INFO: `/v3/users/aa3ad810b8d444368e324475d3291605` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:09.706745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e785b3-a225-4777-b9a7-9f2c37d53e7a None admin] RBAC: Authorizing `identity:get_user(user_id=aa3ad810b8d444368e324475d3291605)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:09.708185 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e785b3-a225-4777-b9a7-9f2c37d53e7a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:09.709874 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 577/1143] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:06:09 2022] GET /identity/v3/users/aa3ad810b8d444368e324475d3291605 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:09.715736 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0207ec-d908-4714-bef7-7c21e3e083cb None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:09.740508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:09.812301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0207ec-d908-4714-bef7-7c21e3e083cb None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:09.845687 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f87444-a76f-44e8-b361-b6f6d827a71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:09.846322 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f87444-a76f-44e8-b361-b6f6d827a71c None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:09.846604 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f87444-a76f-44e8-b361-b6f6d827a71c None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:09.846881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f87444-a76f-44e8-b361-b6f6d827a71c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:09.912352 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f87444-a76f-44e8-b361-b6f6d827a71c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:09.913756 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f87444-a76f-44e8-b361-b6f6d827a71c None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:09.936499 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:10.009613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 567/1144] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:06:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:06:10.015500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e785b3-a225-4777-b9a7-9f2c37d53e7a None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:10.117833 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e785b3-a225-4777-b9a7-9f2c37d53e7a None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:10.146688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:10.153302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7c0ec1-063b-4353-852f-22826bfa73f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:10.153936 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c0ec1-063b-4353-852f-22826bfa73f9 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:10.154213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c0ec1-063b-4353-852f-22826bfa73f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:10.154486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c0ec1-063b-4353-852f-22826bfa73f9 None admin] PATH_INFO: `/v3/projects/7c8266998c5844779e8b43267118c7dc` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:10.155628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c0ec1-063b-4353-852f-22826bfa73f9 None admin] RBAC: Authorizing `identity:get_project(project_id=7c8266998c5844779e8b43267118c7dc)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:10.157078 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7c0ec1-063b-4353-852f-22826bfa73f9 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:10.158736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 578/1145] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:06:10 2022] GET /identity/v3/projects/7c8266998c5844779e8b43267118c7dc => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:10.165114 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f87444-a76f-44e8-b361-b6f6d827a71c None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:10.260531 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f87444-a76f-44e8-b361-b6f6d827a71c None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:10.282724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:10.293408 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d3b62-f7e0-44a7-a646-87914d69d0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:10.294013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d3b62-f7e0-44a7-a646-87914d69d0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:10.294288 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d3b62-f7e0-44a7-a646-87914d69d0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:10.294557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d3b62-f7e0-44a7-a646-87914d69d0d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:10.295329 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d3b62-f7e0-44a7-a646-87914d69d0d1 None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=7c8266998c5844779e8b43267118c7dc, user.id=aa3ad810b8d444368e324475d3291605) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:06:10.295633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d3b62-f7e0-44a7-a646-87914d69d0d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:10.297077 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587d3b62-f7e0-44a7-a646-87914d69d0d1 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:10.302779 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 568/1146] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:06:10 2022] GET /identity/v3/role_assignments?user.id=aa3ad810b8d444368e324475d3291605&scope.project.id=7c8266998c5844779e8b43267118c7dc&role.id=35205394c8af4c52a61b12f92ba16644 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:12.261390 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956f2d4-2660-475d-81c6-47d4ca0df890 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:12.261942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956f2d4-2660-475d-81c6-47d4ca0df890 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:12.261942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956f2d4-2660-475d-81c6-47d4ca0df890 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:12.263273 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 579/1147] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:06:12.269065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567ab94-bcae-4ed7-9d88-284479fe45a5 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:12.269485 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567ab94-bcae-4ed7-9d88-284479fe45a5 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:12.269613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6567ab94-bcae-4ed7-9d88-284479fe45a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:12.287786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-6567ab94-bcae-4ed7-9d88-284479fe45a5 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:06:12.289033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-6567ab94-bcae-4ed7-9d88-284479fe45a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:06:12.353214 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 569/1148] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:06:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:06:12.375530 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956f2d4-2660-475d-81c6-47d4ca0df890 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:12.475151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956f2d4-2660-475d-81c6-47d4ca0df890 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:12.509425 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4409570-e06c-4f78-832b-5584be7b0f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:12.510046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4409570-e06c-4f78-832b-5584be7b0f4c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:12.510323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4409570-e06c-4f78-832b-5584be7b0f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:12.510593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4409570-e06c-4f78-832b-5584be7b0f4c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:12.520421 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4409570-e06c-4f78-832b-5584be7b0f4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:06:12.525156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-e4409570-e06c-4f78-832b-5584be7b0f4c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:06:12.525156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:06:12.525156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:06:12.525156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:06:12.525156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:06:12.525156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:06:12.525156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:06:12.525156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:06:12.525156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:06:12.525156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:06:12.525156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:06:12.525156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:06:12.525156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:06:12.525156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:06:12.525156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:06:12.525156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:06:12.526240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:06:12.526240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:06:12.526240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:06:12.526240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:06:12.526240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:06:12.526240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:06:12.526240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:06:12.526240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:06:12.526240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:06:12.526240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:06:12.526240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:06:12.526240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:06:12.526240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:06:12.526240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:06:12.526240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:06:12.526240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:06:12.527257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:06:12.527257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:06:12.527257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:06:12.527257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:06:12.527257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:06:12.527257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:06:12.527257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:06:12.527257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:06:12.527257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:06:12.527257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:06:12.527257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:06:12.527257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:06:12.527257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 580/1149] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:06:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:06:12.532385 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567ab94-bcae-4ed7-9d88-284479fe45a5 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:12.628949 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6567ab94-bcae-4ed7-9d88-284479fe45a5 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:12.646416 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:12.663827 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e301e-c2fa-466e-b350-639596e7368a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:12.664445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e301e-c2fa-466e-b350-639596e7368a None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:12.664764 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e301e-c2fa-466e-b350-639596e7368a None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:12.665040 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54e301e-c2fa-466e-b350-639596e7368a None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:12.665754 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e301e-c2fa-466e-b350-639596e7368a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:06:12.666053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e301e-c2fa-466e-b350-639596e7368a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:12.667419 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54e301e-c2fa-466e-b350-639596e7368a None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:12.675957 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 570/1150] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:06:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:12.681783 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4409570-e06c-4f78-832b-5584be7b0f4c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:12.739898 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:12.780041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4409570-e06c-4f78-832b-5584be7b0f4c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:12.812590 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a02c2c-c44d-4239-9c0f-20a62e568eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:12.813244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a02c2c-c44d-4239-9c0f-20a62e568eea None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:12.813527 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a02c2c-c44d-4239-9c0f-20a62e568eea None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:12.813824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a02c2c-c44d-4239-9c0f-20a62e568eea None admin] PATH_INFO: `/v3/users/aa3ad810b8d444368e324475d3291605` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:12.815245 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a02c2c-c44d-4239-9c0f-20a62e568eea None admin] RBAC: Authorizing `identity:get_user(user_id=aa3ad810b8d444368e324475d3291605)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:12.816681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a02c2c-c44d-4239-9c0f-20a62e568eea None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:12.818267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 581/1151] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:06:12 2022] GET /identity/v3/users/aa3ad810b8d444368e324475d3291605 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:12.824165 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e301e-c2fa-466e-b350-639596e7368a None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:12.921797 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54e301e-c2fa-466e-b350-639596e7368a None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:12.955074 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3ee13-ae8e-4278-a706-f6b281132a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:12.956230 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3ee13-ae8e-4278-a706-f6b281132a1a None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:12.956512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3ee13-ae8e-4278-a706-f6b281132a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:12.956815 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3ee13-ae8e-4278-a706-f6b281132a1a None admin] PATH_INFO: `/v3/projects/7c8266998c5844779e8b43267118c7dc` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:12.957966 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3ee13-ae8e-4278-a706-f6b281132a1a None admin] RBAC: Authorizing `identity:get_project(project_id=7c8266998c5844779e8b43267118c7dc)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:12.959400 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3ee13-ae8e-4278-a706-f6b281132a1a None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:12.961090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 571/1152] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:06:12 2022] GET /identity/v3/projects/7c8266998c5844779e8b43267118c7dc => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:12.967199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a02c2c-c44d-4239-9c0f-20a62e568eea None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:13.065138 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a02c2c-c44d-4239-9c0f-20a62e568eea None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:13.089843 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:13.099432 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9912534-a53e-44b7-a4cb-58b39abfed6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:13.100088 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9912534-a53e-44b7-a4cb-58b39abfed6b None admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:13.100372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9912534-a53e-44b7-a4cb-58b39abfed6b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:13.100665 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9912534-a53e-44b7-a4cb-58b39abfed6b None admin] PATH_INFO: `/v3/projects/7c8266998c5844779e8b43267118c7dc/users/aa3ad810b8d444368e324475d3291605/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:13.102584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9912534-a53e-44b7-a4cb-58b39abfed6b None admin] RBAC: Authorizing `identity:create_grant(project_id=7c8266998c5844779e8b43267118c7dc, user_id=aa3ad810b8d444368e324475d3291605, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:13.104003 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9912534-a53e-44b7-a4cb-58b39abfed6b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:13.116712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 582/1153] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:06:12 2022] PUT /identity/v3/projects/7c8266998c5844779e8b43267118c7dc/users/aa3ad810b8d444368e324475d3291605/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:06:15.074480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582de562-48d9-43a0-9156-80bf072e6bdd None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:15.074980 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582de562-48d9-43a0-9156-80bf072e6bdd None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:15.074980 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582de562-48d9-43a0-9156-80bf072e6bdd None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:15.076286 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 572/1154] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:06:15.081944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88025f4-b20c-4e15-8083-dd11ccffa962 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:15.082365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88025f4-b20c-4e15-8083-dd11ccffa962 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:15.082469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88025f4-b20c-4e15-8083-dd11ccffa962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:15.100040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c88025f4-b20c-4e15-8083-dd11ccffa962 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:06:15.101229 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c88025f4-b20c-4e15-8083-dd11ccffa962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:06:15.172868 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 583/1155] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:06:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:06:15.196082 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582de562-48d9-43a0-9156-80bf072e6bdd None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:15.244926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:15.294769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582de562-48d9-43a0-9156-80bf072e6bdd None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:15.327149 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd72ed-1ceb-4322-bc4a-21544f91dc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:15.327708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd72ed-1ceb-4322-bc4a-21544f91dc20 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:15.327983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd72ed-1ceb-4322-bc4a-21544f91dc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:15.328249 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd72ed-1ceb-4322-bc4a-21544f91dc20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:15.338188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd72ed-1ceb-4322-bc4a-21544f91dc20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:06:15.342933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-08bd72ed-1ceb-4322-bc4a-21544f91dc20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:06:15.342933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:06:15.342933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:06:15.342933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:06:15.342933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:06:15.342933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:06:15.342933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:06:15.342933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:06:15.342933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:06:15.342933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:06:15.342933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:06:15.342933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:06:15.342933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:06:15.342933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:06:15.342933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:06:15.342933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:06:15.344022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:06:15.344022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:06:15.344022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:06:15.344022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:06:15.344022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:06:15.344022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:06:15.344022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:06:15.344022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:06:15.344022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:06:15.344022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:06:15.344022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:06:15.344022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:06:15.344022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:06:15.344022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:06:15.344022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:06:15.344022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:06:15.345198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:06:15.345198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:06:15.345198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:06:15.345198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:06:15.345198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:06:15.345198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:06:15.345198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:06:15.345198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:06:15.345198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:06:15.345198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:06:15.345198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:06:15.345198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:06:15.345198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 573/1156] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:06:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:06:15.350137 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88025f4-b20c-4e15-8083-dd11ccffa962 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:15.445732 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88025f4-b20c-4e15-8083-dd11ccffa962 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:15.461590 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:15.479957 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47bae6-5e47-49ec-94cf-a864fba65edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:15.480580 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47bae6-5e47-49ec-94cf-a864fba65edf None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:15.480890 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47bae6-5e47-49ec-94cf-a864fba65edf None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:15.481164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47bae6-5e47-49ec-94cf-a864fba65edf None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:15.481881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb47bae6-5e47-49ec-94cf-a864fba65edf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:06:15.482181 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb47bae6-5e47-49ec-94cf-a864fba65edf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:15.483569 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb47bae6-5e47-49ec-94cf-a864fba65edf None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:15.495575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 584/1157] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:06:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:15.501458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd72ed-1ceb-4322-bc4a-21544f91dc20 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:15.601300 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd72ed-1ceb-4322-bc4a-21544f91dc20 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:15.609720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:15.634505 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b7327-4dca-46f4-9124-7358a4e8e1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:15.635136 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b7327-4dca-46f4-9124-7358a4e8e1cd None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:15.635417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b7327-4dca-46f4-9124-7358a4e8e1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:15.635692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b7327-4dca-46f4-9124-7358a4e8e1cd None admin] PATH_INFO: `/v3/users/aa3ad810b8d444368e324475d3291605` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:15.636873 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b7327-4dca-46f4-9124-7358a4e8e1cd None admin] RBAC: Authorizing `identity:get_user(user_id=aa3ad810b8d444368e324475d3291605)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:15.638308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b7327-4dca-46f4-9124-7358a4e8e1cd None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:15.639905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 574/1158] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:06:15 2022] GET /identity/v3/users/aa3ad810b8d444368e324475d3291605 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:15.645793 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47bae6-5e47-49ec-94cf-a864fba65edf None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:15.745309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47bae6-5e47-49ec-94cf-a864fba65edf None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:15.777739 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69894d-58c8-458f-aa0e-6f1a92a5f361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:15.778349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69894d-58c8-458f-aa0e-6f1a92a5f361 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:15.778623 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69894d-58c8-458f-aa0e-6f1a92a5f361 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:15.778896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb69894d-58c8-458f-aa0e-6f1a92a5f361 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:15.823869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:15.846027 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69894d-58c8-458f-aa0e-6f1a92a5f361 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:15.847487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb69894d-58c8-458f-aa0e-6f1a92a5f361 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:15.943343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 585/1159] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:06:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:06:15.949326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b7327-4dca-46f4-9124-7358a4e8e1cd None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:16.046079 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b7327-4dca-46f4-9124-7358a4e8e1cd None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:16.078077 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da87b6c9-a6d3-405d-ba2e-19548d9d2a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:16.078693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87b6c9-a6d3-405d-ba2e-19548d9d2a1d None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:16.078982 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87b6c9-a6d3-405d-ba2e-19548d9d2a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:16.079256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da87b6c9-a6d3-405d-ba2e-19548d9d2a1d None admin] PATH_INFO: `/v3/projects/7c8266998c5844779e8b43267118c7dc` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:16.080408 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87b6c9-a6d3-405d-ba2e-19548d9d2a1d None admin] RBAC: Authorizing `identity:get_project(project_id=7c8266998c5844779e8b43267118c7dc)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:16.082504 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da87b6c9-a6d3-405d-ba2e-19548d9d2a1d None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:16.084154 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 575/1160] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:06:15 2022] GET /identity/v3/projects/7c8266998c5844779e8b43267118c7dc => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:16.090293 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69894d-58c8-458f-aa0e-6f1a92a5f361 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:16.166295 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:16.187066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb69894d-58c8-458f-aa0e-6f1a92a5f361 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:16.220587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c99aae-3d24-4ad0-8baa-98b78f1e33ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:16.221737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c99aae-3d24-4ad0-8baa-98b78f1e33ee None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:16.222017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c99aae-3d24-4ad0-8baa-98b78f1e33ee None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:16.222286 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c99aae-3d24-4ad0-8baa-98b78f1e33ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:16.223061 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c99aae-3d24-4ad0-8baa-98b78f1e33ee None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=7c8266998c5844779e8b43267118c7dc, user.id=aa3ad810b8d444368e324475d3291605) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:06:16.223366 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c99aae-3d24-4ad0-8baa-98b78f1e33ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:16.224809 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c99aae-3d24-4ad0-8baa-98b78f1e33ee None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:16.230460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 586/1161] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:06:16 2022] GET /identity/v3/role_assignments?user.id=aa3ad810b8d444368e324475d3291605&scope.project.id=7c8266998c5844779e8b43267118c7dc&role.id=35205394c8af4c52a61b12f92ba16644 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:18.259319 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392fa3b-7b8d-4970-bf38-988f6d0eaa33 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:18.259895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392fa3b-7b8d-4970-bf38-988f6d0eaa33 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:18.259895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392fa3b-7b8d-4970-bf38-988f6d0eaa33 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:18.261213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 576/1162] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:06:18.266303 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47722f-169c-4158-aebd-2526fe71ff81 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:18.266813 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47722f-169c-4158-aebd-2526fe71ff81 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:18.266904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47722f-169c-4158-aebd-2526fe71ff81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:18.285976 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-df47722f-169c-4158-aebd-2526fe71ff81 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:06:18.287154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-df47722f-169c-4158-aebd-2526fe71ff81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:06:18.355479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 587/1163] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:06:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:06:18.378499 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392fa3b-7b8d-4970-bf38-988f6d0eaa33 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:18.427116 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:18.484512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392fa3b-7b8d-4970-bf38-988f6d0eaa33 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:18.521673 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ba83e-76bd-4a4e-9d37-9cdf79978c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:18.522206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ba83e-76bd-4a4e-9d37-9cdf79978c8d None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:18.522485 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ba83e-76bd-4a4e-9d37-9cdf79978c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:18.522760 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ba83e-76bd-4a4e-9d37-9cdf79978c8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:18.593886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ba83e-76bd-4a4e-9d37-9cdf79978c8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:18.595333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ba83e-76bd-4a4e-9d37-9cdf79978c8d None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:18.638007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:18.693314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 577/1164] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:06:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:06:18.699584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47722f-169c-4158-aebd-2526fe71ff81 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:18.770954 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:18.801077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47722f-169c-4158-aebd-2526fe71ff81 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:18.836629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ebe252-afa6-4f76-9ba3-6f55c8034008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:18.837734 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebe252-afa6-4f76-9ba3-6f55c8034008 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:18.838206 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebe252-afa6-4f76-9ba3-6f55c8034008 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:18.838692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ebe252-afa6-4f76-9ba3-6f55c8034008 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:18.846123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebe252-afa6-4f76-9ba3-6f55c8034008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:18.847548 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ebe252-afa6-4f76-9ba3-6f55c8034008 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:18.861736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-e5ebe252-afa6-4f76-9ba3-6f55c8034008 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 02:06:18.861736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:06:18.861736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 04 02:06:18.861736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 04 02:06:18.861736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:06:18.861736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:06:18.861736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:06:18.861736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:06:18.861736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 04 02:06:18.861736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:06:18.861736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:06:18.861736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:06:18.861736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:06:18.861736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:06:18.861736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:06:18.861736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:06:18.861736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:06:18.863014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:06:18.863014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:06:18.863014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:06:18.863014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:06:18.863014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:06:18.863014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:06:18.863014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:06:18.863014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:06:18.863014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:06:18.863014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:06:18.863014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:06:18.863014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:06:18.863014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:06:18.863014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:06:18.863014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:06:18.863014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:06:18.864086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:06:18.864086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:06:18.864086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:06:18.864086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:06:18.864086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:06:18.864086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:06:18.864086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:06:18.864086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:06:18.864086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:06:18.864086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:06:18.864086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 04 02:06:18.864086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:06:18.864086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 04 02:06:18.864086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:06:18.864086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 588/1165] 10.222.0.45 () {56 vars in 1117 bytes} [Thu Aug 4 02:06:18 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:06:18.870169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ba83e-76bd-4a4e-9d37-9cdf79978c8d None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:18.973691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ba83e-76bd-4a4e-9d37-9cdf79978c8d None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:19.006855 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741a17d-72df-4d70-9f7c-3c114f0d5f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:19.007459 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741a17d-72df-4d70-9f7c-3c114f0d5f79 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:19.007737 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741a17d-72df-4d70-9f7c-3c114f0d5f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:19.008007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741a17d-72df-4d70-9f7c-3c114f0d5f79 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:19.008710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741a17d-72df-4d70-9f7c-3c114f0d5f79 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:06:19.009012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741a17d-72df-4d70-9f7c-3c114f0d5f79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:19.010369 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741a17d-72df-4d70-9f7c-3c114f0d5f79 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:19.017253 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 578/1166] 10.222.0.45 () {56 vars in 1115 bytes} [Thu Aug 4 02:06:18 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:21.005698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c77d-2b91-47d5-a4cb-9145ffa42f45 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:21.006208 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c77d-2b91-47d5-a4cb-9145ffa42f45 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:21.006208 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e362c77d-2b91-47d5-a4cb-9145ffa42f45 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:21.007511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 589/1167] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:06:21.012831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9be34f-5114-4a8b-ad65-892b4778d0c6 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:21.013273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9be34f-5114-4a8b-ad65-892b4778d0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:21.013361 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9be34f-5114-4a8b-ad65-892b4778d0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:21.031539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-3b9be34f-5114-4a8b-ad65-892b4778d0c6 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:06:21.032731 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-3b9be34f-5114-4a8b-ad65-892b4778d0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:06:21.034611 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:21.098959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 579/1168] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:06:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:06:21.121873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362c77d-2b91-47d5-a4cb-9145ffa42f45 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:21.227044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e362c77d-2b91-47d5-a4cb-9145ffa42f45 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:21.255669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:21.260970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2759056f-24c8-4b26-bd51-8bc5256b8fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:21.261525 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759056f-24c8-4b26-bd51-8bc5256b8fc6 None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:21.261806 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759056f-24c8-4b26-bd51-8bc5256b8fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:21.262082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2759056f-24c8-4b26-bd51-8bc5256b8fc6 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:21.262786 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759056f-24c8-4b26-bd51-8bc5256b8fc6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:21.264137 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2759056f-24c8-4b26-bd51-8bc5256b8fc6 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:21.279294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 590/1169] 10.222.0.45 () {60 vars in 1138 bytes} [Thu Aug 4 02:06:21 2022] POST /identity/v3/domains => generated 267 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:06:23.308260 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e3d56-5c92-4775-bb59-14b6676cab7d None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:23.308822 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e3d56-5c92-4775-bb59-14b6676cab7d None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:23.308970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e3d56-5c92-4775-bb59-14b6676cab7d None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:23.310334 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 580/1170] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:06:23.316415 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3eb0e-a333-41ac-b6b3-31e8718bff4a None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:23.316863 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3eb0e-a333-41ac-b6b3-31e8718bff4a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:23.316995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea3eb0e-a333-41ac-b6b3-31e8718bff4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:23.334639 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-2ea3eb0e-a333-41ac-b6b3-31e8718bff4a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:06:23.335825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-2ea3eb0e-a333-41ac-b6b3-31e8718bff4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:06:23.401835 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 591/1171] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:06:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:06:23.425086 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e3d56-5c92-4775-bb59-14b6676cab7d None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:23.523197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e3d56-5c92-4775-bb59-14b6676cab7d None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:23.556386 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e508c5-e1ab-490d-aac0-0b1348cfb67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:23.557052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e508c5-e1ab-490d-aac0-0b1348cfb67e None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:23.557335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e508c5-e1ab-490d-aac0-0b1348cfb67e None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:23.557610 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e508c5-e1ab-490d-aac0-0b1348cfb67e None admin] PATH_INFO: `/v3/domains/f49a37d6025a4ba692936b8a83ffd4b7` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:23.563984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:23.568975 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e508c5-e1ab-490d-aac0-0b1348cfb67e None admin] RBAC: Authorizing `identity:get_domain(domain_id=f49a37d6025a4ba692936b8a83ffd4b7)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:23.570427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e508c5-e1ab-490d-aac0-0b1348cfb67e None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:23.572085 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 581/1172] 10.222.0.45 () {56 vars in 1183 bytes} [Thu Aug 4 02:06:23 2022] GET /identity/v3/domains/f49a37d6025a4ba692936b8a83ffd4b7 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:23.577983 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3eb0e-a333-41ac-b6b3-31e8718bff4a None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:23.679943 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea3eb0e-a333-41ac-b6b3-31e8718bff4a None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:23.702825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:23.715111 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2987cd-6d56-4046-a852-15d3807edd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:23.715800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2987cd-6d56-4046-a852-15d3807edd9b None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:23.716013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2987cd-6d56-4046-a852-15d3807edd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:23.716289 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2987cd-6d56-4046-a852-15d3807edd9b None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:23.717048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2987cd-6d56-4046-a852-15d3807edd9b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:23.718439 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2987cd-6d56-4046-a852-15d3807edd9b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:23.745252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 592/1173] 10.222.0.45 () {60 vars in 1141 bytes} [Thu Aug 4 02:06:23 2022] POST /identity/v3/projects => generated 361 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:06:25.801365 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5bfe6-e654-4a5b-821d-f40c0c29f040 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:25.801938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5bfe6-e654-4a5b-821d-f40c0c29f040 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:25.801938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a5bfe6-e654-4a5b-821d-f40c0c29f040 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:25.803276 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 582/1174] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:06:25.810252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4c7e3-79cd-46f3-b0d6-1051a7aecbee None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:25.810797 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4c7e3-79cd-46f3-b0d6-1051a7aecbee None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:25.810797 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c4c7e3-79cd-46f3-b0d6-1051a7aecbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:25.830224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-b7c4c7e3-79cd-46f3-b0d6-1051a7aecbee None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:06:25.832032 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b7c4c7e3-79cd-46f3-b0d6-1051a7aecbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:06:25.903404 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 593/1175] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:06:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:06:25.927807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5bfe6-e654-4a5b-821d-f40c0c29f040 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:26.026839 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a5bfe6-e654-4a5b-821d-f40c0c29f040 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:26.058695 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf93e3d-0df3-4df8-82a5-de33c170dba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:26.059323 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf93e3d-0df3-4df8-82a5-de33c170dba1 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:26.059604 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf93e3d-0df3-4df8-82a5-de33c170dba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:26.059880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf93e3d-0df3-4df8-82a5-de33c170dba1 None admin] PATH_INFO: `/v3/domains/f49a37d6025a4ba692936b8a83ffd4b7` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:26.061051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf93e3d-0df3-4df8-82a5-de33c170dba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f49a37d6025a4ba692936b8a83ffd4b7)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:26.062482 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf93e3d-0df3-4df8-82a5-de33c170dba1 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:26.064060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 583/1176] 10.222.0.45 () {56 vars in 1183 bytes} [Thu Aug 4 02:06:25 2022] GET /identity/v3/domains/f49a37d6025a4ba692936b8a83ffd4b7 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:26.070447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4c7e3-79cd-46f3-b0d6-1051a7aecbee None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:26.167036 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c4c7e3-79cd-46f3-b0d6-1051a7aecbee None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:26.169321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:26.201733 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d894de-6468-418e-9d50-75a94e163232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:26.202260 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d894de-6468-418e-9d50-75a94e163232 None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:26.202535 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d894de-6468-418e-9d50-75a94e163232 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:26.202805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d894de-6468-418e-9d50-75a94e163232 None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:26.203489 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d894de-6468-418e-9d50-75a94e163232 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:26.204921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d894de-6468-418e-9d50-75a94e163232 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:26.225911 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 594/1177] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:06:26 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:06:28.230364 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab3bde-e53c-46e1-8656-d71965f23bab None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:28.230926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab3bde-e53c-46e1-8656-d71965f23bab None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:28.230926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab3bde-e53c-46e1-8656-d71965f23bab None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:28.232258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 584/1178] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:06:28.237622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed132e-2e2d-4cb8-9c24-1917eb537904 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:28.238013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed132e-2e2d-4cb8-9c24-1917eb537904 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:28.238193 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed132e-2e2d-4cb8-9c24-1917eb537904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:28.256272 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-04ed132e-2e2d-4cb8-9c24-1917eb537904 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:06:28.257480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-04ed132e-2e2d-4cb8-9c24-1917eb537904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:06:28.323585 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 595/1179] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:06:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:06:28.346691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab3bde-e53c-46e1-8656-d71965f23bab None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:28.411032 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:28.447235 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab3bde-e53c-46e1-8656-d71965f23bab None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:28.478997 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b394cb4-9701-4cc6-a4b7-dcc51401f4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:28.479628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b394cb4-9701-4cc6-a4b7-dcc51401f4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:28.479904 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b394cb4-9701-4cc6-a4b7-dcc51401f4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:28.480181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b394cb4-9701-4cc6-a4b7-dcc51401f4a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:28.491456 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b394cb4-9701-4cc6-a4b7-dcc51401f4a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:06:28.496189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-8b394cb4-9701-4cc6-a4b7-dcc51401f4a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:06:28.496189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:06:28.496189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:06:28.496189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:06:28.496189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:06:28.496189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:06:28.496189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:06:28.496189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:06:28.496189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:06:28.496189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:06:28.496189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:06:28.496189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:06:28.496189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:06:28.496189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:06:28.496189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:06:28.496189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:06:28.497292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:06:28.497292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:06:28.497292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:06:28.497292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:06:28.497292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:06:28.497292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:06:28.497292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:06:28.497292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:06:28.497292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:06:28.497292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:06:28.497292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:06:28.497292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:06:28.497292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:06:28.497292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:06:28.497292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:06:28.497292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:06:28.498470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:06:28.498470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:06:28.498470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:06:28.498470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:06:28.498470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:06:28.498470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:06:28.498470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:06:28.498470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:06:28.498470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:06:28.498470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:06:28.498470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:06:28.498470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:06:28.498470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 585/1180] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:06:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:06:28.504421 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed132e-2e2d-4cb8-9c24-1917eb537904 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:28.591187 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:28.603753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed132e-2e2d-4cb8-9c24-1917eb537904 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:28.637588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a4436-d812-4aae-9130-dbd3fbf2a78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:28.638192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a4436-d812-4aae-9130-dbd3fbf2a78d None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:28.638469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a4436-d812-4aae-9130-dbd3fbf2a78d None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:28.638739 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a4436-d812-4aae-9130-dbd3fbf2a78d None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:28.639441 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a4436-d812-4aae-9130-dbd3fbf2a78d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:06:28.639740 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a4436-d812-4aae-9130-dbd3fbf2a78d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:28.641132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a4436-d812-4aae-9130-dbd3fbf2a78d None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:28.649444 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 596/1181] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:06:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:28.655382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b394cb4-9701-4cc6-a4b7-dcc51401f4a7 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:28.752028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b394cb4-9701-4cc6-a4b7-dcc51401f4a7 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:28.775653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:28.785246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993948a-e5e4-465b-a6c0-f421a30ed6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:28.785875 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993948a-e5e4-465b-a6c0-f421a30ed6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:28.786154 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993948a-e5e4-465b-a6c0-f421a30ed6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:28.786425 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3993948a-e5e4-465b-a6c0-f421a30ed6d6 None admin] PATH_INFO: `/v3/users/fc43cf43adce470681b23976c9412629` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:28.799815 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993948a-e5e4-465b-a6c0-f421a30ed6d6 None admin] RBAC: Authorizing `identity:get_user(user_id=fc43cf43adce470681b23976c9412629)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:28.801291 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3993948a-e5e4-465b-a6c0-f421a30ed6d6 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:28.802896 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 586/1182] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:06:28 2022] GET /identity/v3/users/fc43cf43adce470681b23976c9412629 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:28.808704 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a4436-d812-4aae-9130-dbd3fbf2a78d None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:28.909180 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a4436-d812-4aae-9130-dbd3fbf2a78d None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:28.942577 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304a2ae-c66a-42fa-a961-401f0f97f352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:28.943222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304a2ae-c66a-42fa-a961-401f0f97f352 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:28.943536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304a2ae-c66a-42fa-a961-401f0f97f352 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:28.943869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304a2ae-c66a-42fa-a961-401f0f97f352 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:28.965798 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:29.011403 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3304a2ae-c66a-42fa-a961-401f0f97f352 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:29.012796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3304a2ae-c66a-42fa-a961-401f0f97f352 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:29.107645 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 597/1183] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:06:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:06:29.113599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993948a-e5e4-465b-a6c0-f421a30ed6d6 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:29.214838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3993948a-e5e4-465b-a6c0-f421a30ed6d6 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:29.247384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec283220-4c90-44ab-84e7-8a5cb94acd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:29.248009 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec283220-4c90-44ab-84e7-8a5cb94acd06 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:29.248288 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec283220-4c90-44ab-84e7-8a5cb94acd06 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:29.248560 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec283220-4c90-44ab-84e7-8a5cb94acd06 None admin] PATH_INFO: `/v3/projects/ab17699c2b2b4791843c03ca3f87d6b8` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:29.249718 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec283220-4c90-44ab-84e7-8a5cb94acd06 None admin] RBAC: Authorizing `identity:get_project(project_id=ab17699c2b2b4791843c03ca3f87d6b8)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:29.251145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec283220-4c90-44ab-84e7-8a5cb94acd06 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:29.252863 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 587/1184] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:06:29 2022] GET /identity/v3/projects/ab17699c2b2b4791843c03ca3f87d6b8 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:29.258920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304a2ae-c66a-42fa-a961-401f0f97f352 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:29.309487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:29.355216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304a2ae-c66a-42fa-a961-401f0f97f352 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:29.386903 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733d9d3-b81a-4cbf-8495-b18577231c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:29.387496 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733d9d3-b81a-4cbf-8495-b18577231c98 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:29.387770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733d9d3-b81a-4cbf-8495-b18577231c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:29.388037 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733d9d3-b81a-4cbf-8495-b18577231c98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:29.388829 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733d9d3-b81a-4cbf-8495-b18577231c98 None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=ab17699c2b2b4791843c03ca3f87d6b8, user.id=fc43cf43adce470681b23976c9412629) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:06:29.389134 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733d9d3-b81a-4cbf-8495-b18577231c98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:29.390538 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733d9d3-b81a-4cbf-8495-b18577231c98 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:29.396234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 598/1185] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:06:29 2022] GET /identity/v3/role_assignments?user.id=fc43cf43adce470681b23976c9412629&scope.project.id=ab17699c2b2b4791843c03ca3f87d6b8&role.id=35205394c8af4c52a61b12f92ba16644 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:31.351727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5303e3-264c-487b-bb3d-ea54c9bf916a None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:31.352329 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5303e3-264c-487b-bb3d-ea54c9bf916a None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:31.352329 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5303e3-264c-487b-bb3d-ea54c9bf916a None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:31.353672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 588/1186] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:06:31.358847 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772c6f6-35c0-4bf4-a20d-2dc12fa6e579 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:31.359275 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772c6f6-35c0-4bf4-a20d-2dc12fa6e579 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:31.359398 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772c6f6-35c0-4bf4-a20d-2dc12fa6e579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:31.377037 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-9772c6f6-35c0-4bf4-a20d-2dc12fa6e579 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:06:31.378288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9772c6f6-35c0-4bf4-a20d-2dc12fa6e579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:06:31.442203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 599/1187] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:06:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:06:31.464444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5303e3-264c-487b-bb3d-ea54c9bf916a None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:31.516588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:31.563427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5303e3-264c-487b-bb3d-ea54c9bf916a None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:31.597193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053e506-669f-4095-b71a-83b4ab4b9cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:31.597821 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053e506-669f-4095-b71a-83b4ab4b9cde None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:31.598102 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053e506-669f-4095-b71a-83b4ab4b9cde None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:31.598378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053e506-669f-4095-b71a-83b4ab4b9cde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:31.608230 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053e506-669f-4095-b71a-83b4ab4b9cde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:06:31.613033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-5053e506-669f-4095-b71a-83b4ab4b9cde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:06:31.613033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:06:31.613033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:06:31.613033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:06:31.613033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:06:31.613033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:06:31.613033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:06:31.613033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:06:31.613033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:06:31.613033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:06:31.613033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:06:31.613033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:06:31.613033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:06:31.613033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:06:31.613033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:06:31.613033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:06:31.614181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:06:31.614181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:06:31.614181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:06:31.614181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:06:31.614181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:06:31.614181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:06:31.614181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:06:31.614181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:06:31.614181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:06:31.614181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:06:31.614181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:06:31.614181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:06:31.614181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:06:31.614181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:06:31.614181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:06:31.614181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:06:31.615231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:06:31.615231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:06:31.615231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:06:31.615231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:06:31.615231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:06:31.615231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:06:31.615231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:06:31.615231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:06:31.615231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:06:31.615231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:06:31.615231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:06:31.615231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:06:31.615231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 589/1188] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:06:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:06:31.620351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772c6f6-35c0-4bf4-a20d-2dc12fa6e579 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:31.666552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:31.718500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772c6f6-35c0-4bf4-a20d-2dc12fa6e579 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:31.752850 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5ae1e-2410-4714-9d13-871508af876e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:31.753465 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5ae1e-2410-4714-9d13-871508af876e None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:31.753745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5ae1e-2410-4714-9d13-871508af876e None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:31.754020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5ae1e-2410-4714-9d13-871508af876e None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:31.754736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5ae1e-2410-4714-9d13-871508af876e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:06:31.755043 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5ae1e-2410-4714-9d13-871508af876e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:31.756409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5ae1e-2410-4714-9d13-871508af876e None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:31.764360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 600/1189] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:06:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:31.770917 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053e506-669f-4095-b71a-83b4ab4b9cde None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:31.865322 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053e506-669f-4095-b71a-83b4ab4b9cde None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:31.878087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:31.899306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b83ef3-9381-4aef-9348-5962c56a0a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:31.899949 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b83ef3-9381-4aef-9348-5962c56a0a59 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:31.900230 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b83ef3-9381-4aef-9348-5962c56a0a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:31.900500 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b83ef3-9381-4aef-9348-5962c56a0a59 None admin] PATH_INFO: `/v3/users/fc43cf43adce470681b23976c9412629` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:31.901715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b83ef3-9381-4aef-9348-5962c56a0a59 None admin] RBAC: Authorizing `identity:get_user(user_id=fc43cf43adce470681b23976c9412629)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:31.903148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b83ef3-9381-4aef-9348-5962c56a0a59 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:31.904771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 590/1190] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:06:31 2022] GET /identity/v3/users/fc43cf43adce470681b23976c9412629 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:31.910806 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5ae1e-2410-4714-9d13-871508af876e None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:32.008999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5ae1e-2410-4714-9d13-871508af876e None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:32.017531 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:32.042418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad805c-0280-46df-a58e-1f64e1cf3ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:32.043053 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad805c-0280-46df-a58e-1f64e1cf3ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:32.043332 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad805c-0280-46df-a58e-1f64e1cf3ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:32.043605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad805c-0280-46df-a58e-1f64e1cf3ce1 None admin] PATH_INFO: `/v3/projects/ab17699c2b2b4791843c03ca3f87d6b8` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:32.044779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad805c-0280-46df-a58e-1f64e1cf3ce1 None admin] RBAC: Authorizing `identity:get_project(project_id=ab17699c2b2b4791843c03ca3f87d6b8)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:32.046211 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad805c-0280-46df-a58e-1f64e1cf3ce1 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:32.047877 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 601/1191] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:06:31 2022] GET /identity/v3/projects/ab17699c2b2b4791843c03ca3f87d6b8 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:32.053853 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b83ef3-9381-4aef-9348-5962c56a0a59 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:32.150977 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b83ef3-9381-4aef-9348-5962c56a0a59 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:32.183506 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d80f548-ebf3-48e4-996a-2550e2d4e15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:32.184159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80f548-ebf3-48e4-996a-2550e2d4e15c None admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:32.184437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80f548-ebf3-48e4-996a-2550e2d4e15c None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:32.184732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80f548-ebf3-48e4-996a-2550e2d4e15c None admin] PATH_INFO: `/v3/projects/ab17699c2b2b4791843c03ca3f87d6b8/users/fc43cf43adce470681b23976c9412629/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:32.186640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80f548-ebf3-48e4-996a-2550e2d4e15c None admin] RBAC: Authorizing `identity:create_grant(project_id=ab17699c2b2b4791843c03ca3f87d6b8, user_id=fc43cf43adce470681b23976c9412629, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:32.188066 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d80f548-ebf3-48e4-996a-2550e2d4e15c None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:32.201095 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 591/1192] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:06:32 2022] PUT /identity/v3/projects/ab17699c2b2b4791843c03ca3f87d6b8/users/fc43cf43adce470681b23976c9412629/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:06:34.202595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a7206-f113-4328-91bb-057b7a191f06 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:34.203137 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a7206-f113-4328-91bb-057b7a191f06 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:34.203137 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a7206-f113-4328-91bb-057b7a191f06 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:34.204730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 602/1193] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:06:34.210333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5aba91-7fb4-4335-9d91-9c517d5a7564 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:34.210838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5aba91-7fb4-4335-9d91-9c517d5a7564 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:34.210933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5aba91-7fb4-4335-9d91-9c517d5a7564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:34.231844 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-4e5aba91-7fb4-4335-9d91-9c517d5a7564 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:06:34.233082 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-4e5aba91-7fb4-4335-9d91-9c517d5a7564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:06:34.273888 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:34.308772 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 592/1194] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:06:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:06:34.332268 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a7206-f113-4328-91bb-057b7a191f06 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:34.432283 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a7206-f113-4328-91bb-057b7a191f06 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:34.465279 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f53df-2441-4867-a54d-90f306119312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:34.465859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f53df-2441-4867-a54d-90f306119312 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:34.466138 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f53df-2441-4867-a54d-90f306119312 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:34.466409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f53df-2441-4867-a54d-90f306119312 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:34.477234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f53df-2441-4867-a54d-90f306119312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:06:34.482012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-798f53df-2441-4867-a54d-90f306119312 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:06:34.482012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:06:34.482012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:06:34.482012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:06:34.482012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:06:34.482012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:06:34.482012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:06:34.482012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:06:34.482012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:06:34.482012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:06:34.482012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:06:34.482012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:06:34.482012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:06:34.482012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:06:34.482012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:06:34.482012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:06:34.483073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:06:34.483073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:06:34.483073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:06:34.483073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:06:34.483073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:06:34.483073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:06:34.483073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:06:34.483073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:06:34.483073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:06:34.483073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:06:34.483073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:06:34.483073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:06:34.483073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:06:34.483073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:06:34.483073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:06:34.483073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:06:34.483931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:06:34.483931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:06:34.483931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:06:34.483931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:06:34.483931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:06:34.483931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:06:34.483931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:06:34.483931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:06:34.483931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:06:34.483931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:06:34.483931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:06:34.483931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:06:34.483931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 603/1195] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:06:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:06:34.489357 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5aba91-7fb4-4335-9d91-9c517d5a7564 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:34.588721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5aba91-7fb4-4335-9d91-9c517d5a7564 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:34.623768 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5700883-db50-41fc-a9b6-89ff7a5a66f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:34.624377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5700883-db50-41fc-a9b6-89ff7a5a66f6 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:34.624682 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5700883-db50-41fc-a9b6-89ff7a5a66f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:34.624959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5700883-db50-41fc-a9b6-89ff7a5a66f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:34.625668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5700883-db50-41fc-a9b6-89ff7a5a66f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:06:34.625974 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5700883-db50-41fc-a9b6-89ff7a5a66f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:34.627342 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5700883-db50-41fc-a9b6-89ff7a5a66f6 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:34.639186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 593/1196] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:06:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:34.645512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f53df-2441-4867-a54d-90f306119312 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:34.680341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:34.748056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f53df-2441-4867-a54d-90f306119312 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:34.781379 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc8cfa-5813-4e3e-b3e2-4aa6f867176b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:34.782019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc8cfa-5813-4e3e-b3e2-4aa6f867176b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:34.782301 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc8cfa-5813-4e3e-b3e2-4aa6f867176b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:34.782580 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc8cfa-5813-4e3e-b3e2-4aa6f867176b None admin] PATH_INFO: `/v3/users/fc43cf43adce470681b23976c9412629` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:34.783761 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc8cfa-5813-4e3e-b3e2-4aa6f867176b None admin] RBAC: Authorizing `identity:get_user(user_id=fc43cf43adce470681b23976c9412629)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:34.785242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dc8cfa-5813-4e3e-b3e2-4aa6f867176b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:34.786879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 604/1197] 10.222.0.45 () {56 vars in 1177 bytes} [Thu Aug 4 02:06:34 2022] GET /identity/v3/users/fc43cf43adce470681b23976c9412629 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:34.792656 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5700883-db50-41fc-a9b6-89ff7a5a66f6 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:34.823891 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:34.892804 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5700883-db50-41fc-a9b6-89ff7a5a66f6 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:34.928577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea8678-7d89-4992-ae02-45bdc7b7c729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:34.929223 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea8678-7d89-4992-ae02-45bdc7b7c729 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:34.929506 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea8678-7d89-4992-ae02-45bdc7b7c729 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:34.929781 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea8678-7d89-4992-ae02-45bdc7b7c729 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:34.996072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea8678-7d89-4992-ae02-45bdc7b7c729 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:34.997527 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea8678-7d89-4992-ae02-45bdc7b7c729 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:35.025963 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:35.091503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 594/1198] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:06:34 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:06:35.097418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc8cfa-5813-4e3e-b3e2-4aa6f867176b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:35.195083 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc8cfa-5813-4e3e-b3e2-4aa6f867176b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:35.197366 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:35.228947 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ee5ea-d816-4b72-b29e-743b823bab22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:35.230018 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ee5ea-d816-4b72-b29e-743b823bab22 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:35.230423 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ee5ea-d816-4b72-b29e-743b823bab22 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:35.230701 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ee5ea-d816-4b72-b29e-743b823bab22 None admin] PATH_INFO: `/v3/projects/ab17699c2b2b4791843c03ca3f87d6b8` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:35.231860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ee5ea-d816-4b72-b29e-743b823bab22 None admin] RBAC: Authorizing `identity:get_project(project_id=ab17699c2b2b4791843c03ca3f87d6b8)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:35.233305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ee5ea-d816-4b72-b29e-743b823bab22 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:35.234970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 605/1199] 10.222.0.45 () {56 vars in 1186 bytes} [Thu Aug 4 02:06:35 2022] GET /identity/v3/projects/ab17699c2b2b4791843c03ca3f87d6b8 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:35.241624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea8678-7d89-4992-ae02-45bdc7b7c729 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:06:35.337760 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea8678-7d89-4992-ae02-45bdc7b7c729 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:06:35.366028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:35.371383 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a27ef2-e3f3-4cff-a528-ef3c261b35ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:06:35.371980 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a27ef2-e3f3-4cff-a528-ef3c261b35ec None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:06:35.372253 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a27ef2-e3f3-4cff-a528-ef3c261b35ec None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:06:35.372520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a27ef2-e3f3-4cff-a528-ef3c261b35ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:06:35.373316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a27ef2-e3f3-4cff-a528-ef3c261b35ec None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=ab17699c2b2b4791843c03ca3f87d6b8, user.id=fc43cf43adce470681b23976c9412629) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:06:35.373617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a27ef2-e3f3-4cff-a528-ef3c261b35ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:06:35.375018 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a27ef2-e3f3-4cff-a528-ef3c261b35ec None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:06:35.381246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 595/1200] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:06:35 2022] GET /identity/v3/role_assignments?user.id=fc43cf43adce470681b23976c9412629&scope.project.id=ab17699c2b2b4791843c03ca3f87d6b8&role.id=35205394c8af4c52a61b12f92ba16644 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:06:45.223157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=14 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:06:45.378689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:30.861595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d7dc8-fc07-42ef-8c34-d15e3e761314 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:30.861595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d7dc8-fc07-42ef-8c34-d15e3e761314 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:30.861595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18d7dc8-fc07-42ef-8c34-d15e3e761314 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:30.862772 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 606/1201] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:07:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:07:30.867229 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62ff1b-09b4-468d-b93a-a60e952228e6 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:30.867683 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62ff1b-09b4-468d-b93a-a60e952228e6 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:30.867783 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b62ff1b-09b4-468d-b93a-a60e952228e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:30.886860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-0b62ff1b-09b4-468d-b93a-a60e952228e6 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:07:30.888464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-0b62ff1b-09b4-468d-b93a-a60e952228e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:07:30.986339 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 596/1202] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:07:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:07:31.016688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d7dc8-fc07-42ef-8c34-d15e3e761314 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:31.149940 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18d7dc8-fc07-42ef-8c34-d15e3e761314 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:31.191097 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a3dd4-816a-4d90-b9a7-c6517fab3a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:31.191852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a3dd4-816a-4d90-b9a7-c6517fab3a2e None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:31.192383 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a3dd4-816a-4d90-b9a7-c6517fab3a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:31.192683 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a3dd4-816a-4d90-b9a7-c6517fab3a2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:31.206452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a3dd4-816a-4d90-b9a7-c6517fab3a2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:31.208188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a3dd4-816a-4d90-b9a7-c6517fab3a2e None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:31.226720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-799a3dd4-816a-4d90-b9a7-c6517fab3a2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:31.226720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:31.226720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:31.226720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:31.226720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:31.226720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:31.226720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:31.226720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:31.226720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:31.226720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:31.226720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:31.226720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:31.226720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:07:31.226720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:07:31.226720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:07:31.227901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:07:31.227901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:31.227901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:31.227901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:31.227901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:31.227901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:31.227901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:31.227901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:31.227901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:31.227901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:31.227901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:31.227901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:31.227901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:31.227901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:31.227901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:31.227901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:31.227901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:31.229118 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:07:31.229118 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:07:31.229118 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:31.229118 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:07:31.229118 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 607/1203] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:31.235562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62ff1b-09b4-468d-b93a-a60e952228e6 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:31.378821 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b62ff1b-09b4-468d-b93a-a60e952228e6 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:31.412597 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:31.419717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ded1cd-db20-4b81-8344-9e28aef4ba91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:31.420539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ded1cd-db20-4b81-8344-9e28aef4ba91 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:31.421035 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ded1cd-db20-4b81-8344-9e28aef4ba91 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:31.421331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ded1cd-db20-4b81-8344-9e28aef4ba91 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:31.422107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ded1cd-db20-4b81-8344-9e28aef4ba91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:31.422441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ded1cd-db20-4b81-8344-9e28aef4ba91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:31.423907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ded1cd-db20-4b81-8344-9e28aef4ba91 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:31.441910 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 597/1204] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:31.447686 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a3dd4-816a-4d90-b9a7-c6517fab3a2e None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:31.513503 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:31.570929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a3dd4-816a-4d90-b9a7-c6517fab3a2e None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:31.610464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b6e79c-6ba7-4175-a301-c569df19cf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:31.611093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6e79c-6ba7-4175-a301-c569df19cf0a None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:31.611378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6e79c-6ba7-4175-a301-c569df19cf0a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:31.611856 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b6e79c-6ba7-4175-a301-c569df19cf0a None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:31.612885 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6e79c-6ba7-4175-a301-c569df19cf0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:31.615932 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b6e79c-6ba7-4175-a301-c569df19cf0a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:31.647743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 608/1205] 10.222.0.45 () {60 vars in 1132 bytes} [Thu Aug 4 02:07:31 2022] POST /identity/v3/users => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:07:33.979696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440bfcb-f510-49da-8189-4775b0ac26f0 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:33.980234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440bfcb-f510-49da-8189-4775b0ac26f0 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:33.980234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d440bfcb-f510-49da-8189-4775b0ac26f0 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:33.981717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 598/1206] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:07:33.988659 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0f152-8940-4af5-82d4-c01eb35963fa None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:33.989169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0f152-8940-4af5-82d4-c01eb35963fa None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:33.989318 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0f152-8940-4af5-82d4-c01eb35963fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:34.010582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-56c0f152-8940-4af5-82d4-c01eb35963fa None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:07:34.011936 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-56c0f152-8940-4af5-82d4-c01eb35963fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:07:34.090895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 609/1207] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:07:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:07:34.117010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440bfcb-f510-49da-8189-4775b0ac26f0 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:34.263137 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d440bfcb-f510-49da-8189-4775b0ac26f0 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:34.310170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36119717-e46b-4d69-9eb2-4c09d005bbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:34.310949 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36119717-e46b-4d69-9eb2-4c09d005bbbe None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:34.311222 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36119717-e46b-4d69-9eb2-4c09d005bbbe None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:34.311533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36119717-e46b-4d69-9eb2-4c09d005bbbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:34.327394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36119717-e46b-4d69-9eb2-4c09d005bbbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:07:34.333529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-36119717-e46b-4d69-9eb2-4c09d005bbbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:07:34.333529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:34.333529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:34.333529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:34.333529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:34.333529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:34.333529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:34.333529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:34.333529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:34.333529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:34.333529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:34.333529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:34.333529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:07:34.333529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:07:34.333529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:07:34.333529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:07:34.334716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:07:34.334716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:07:34.334716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:07:34.334716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:07:34.334716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:34.334716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:34.334716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:34.334716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:34.334716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:34.334716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:34.334716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:34.334716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:34.334716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:34.334716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:34.334716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:34.334716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:34.335629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:34.335629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:34.335629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:34.335629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:34.335629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:07:34.335629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:07:34.335629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:07:34.335629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:07:34.335629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:07:34.335629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:07:34.335629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:07:34.335629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:07:34.335629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 599/1208] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:07:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:34.341599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0f152-8940-4af5-82d4-c01eb35963fa None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:34.390099 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:34.464943 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0f152-8940-4af5-82d4-c01eb35963fa None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:34.512327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d372b37-3f2e-4fc8-831d-b21349d400c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:34.512695 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d372b37-3f2e-4fc8-831d-b21349d400c9 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:34.516383 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d372b37-3f2e-4fc8-831d-b21349d400c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:34.516754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d372b37-3f2e-4fc8-831d-b21349d400c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:34.517759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d372b37-3f2e-4fc8-831d-b21349d400c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:34.518092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d372b37-3f2e-4fc8-831d-b21349d400c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:34.519936 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d372b37-3f2e-4fc8-831d-b21349d400c9 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:34.530968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 610/1209] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:07:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:34.536800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36119717-e46b-4d69-9eb2-4c09d005bbbe None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:34.604756 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:34.680491 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36119717-e46b-4d69-9eb2-4c09d005bbbe None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:34.723836 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c5571-a5f5-4042-abc1-2b23db77550d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:34.726455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c5571-a5f5-4042-abc1-2b23db77550d None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:34.726455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c5571-a5f5-4042-abc1-2b23db77550d None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:34.726455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c5571-a5f5-4042-abc1-2b23db77550d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:34.741364 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c5571-a5f5-4042-abc1-2b23db77550d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:34.742954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c5571-a5f5-4042-abc1-2b23db77550d None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:34.761616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-759c5571-a5f5-4042-abc1-2b23db77550d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:34.761616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:34.761616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:34.761616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:34.761616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:34.761616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:34.761616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:34.761616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:34.761616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:34.761616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:34.761616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:34.761616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:34.761616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:07:34.761616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:07:34.761616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:07:34.762605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:07:34.762605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:34.762605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:34.762605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:34.762605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:34.762605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:34.762605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:34.762605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:34.762605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:34.762605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:34.762605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:34.762605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:34.762605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:34.762605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:34.762605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:34.762605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:34.762605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:34.763522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:07:34.763522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:07:34.763522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:34.763522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:07:34.763522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 600/1210] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:07:34.769571 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d372b37-3f2e-4fc8-831d-b21349d400c9 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:34.900239 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:34.907114 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d372b37-3f2e-4fc8-831d-b21349d400c9 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:34.956593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc61dee-b2a2-42a2-94c5-5f9237bb3005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:34.957555 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc61dee-b2a2-42a2-94c5-5f9237bb3005 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:34.957835 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc61dee-b2a2-42a2-94c5-5f9237bb3005 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:34.958173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc61dee-b2a2-42a2-94c5-5f9237bb3005 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:34.958889 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc61dee-b2a2-42a2-94c5-5f9237bb3005 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:34.959233 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc61dee-b2a2-42a2-94c5-5f9237bb3005 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:34.960652 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc61dee-b2a2-42a2-94c5-5f9237bb3005 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:34.973991 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 611/1211] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:34.979558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c5571-a5f5-4042-abc1-2b23db77550d None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:35.106248 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c5571-a5f5-4042-abc1-2b23db77550d None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:35.107703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:35.151950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226202c-e383-4cb1-acf4-02e3265341b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:35.152792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226202c-e383-4cb1-acf4-02e3265341b8 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:35.153198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226202c-e383-4cb1-acf4-02e3265341b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:35.153475 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226202c-e383-4cb1-acf4-02e3265341b8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:35.165652 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226202c-e383-4cb1-acf4-02e3265341b8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:35.167116 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226202c-e383-4cb1-acf4-02e3265341b8 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:35.181809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-8226202c-e383-4cb1-acf4-02e3265341b8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 02:07:35.181809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:35.181809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:35.181809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:35.181809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:35.181809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:35.181809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:35.181809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:35.181809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:35.181809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:35.181809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:35.181809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:35.181809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:07:35.181809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:07:35.181809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:07:35.181809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:35.183169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:35.184139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:35.184139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:35.184139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:07:35.184139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:07:35.184139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:07:35.184139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:07:35.184139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:07:35.184139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:07:35.184139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 02:07:35.184139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:07:35.184139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 601/1212] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:07:35.189546 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc61dee-b2a2-42a2-94c5-5f9237bb3005 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:35.319773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc61dee-b2a2-42a2-94c5-5f9237bb3005 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:35.368151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43a820-6cd3-4f1c-a82a-2e15b3b555ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:35.368151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43a820-6cd3-4f1c-a82a-2e15b3b555ca None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:35.368151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43a820-6cd3-4f1c-a82a-2e15b3b555ca None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:35.368558 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43a820-6cd3-4f1c-a82a-2e15b3b555ca None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:35.369191 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43a820-6cd3-4f1c-a82a-2e15b3b555ca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:35.370251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43a820-6cd3-4f1c-a82a-2e15b3b555ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:35.371419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43a820-6cd3-4f1c-a82a-2e15b3b555ca None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:35.393215 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 612/1213] 10.222.0.45 () {56 vars in 1143 bytes} [Thu Aug 4 02:07:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:35.399557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226202c-e383-4cb1-acf4-02e3265341b8 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:35.532089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226202c-e383-4cb1-acf4-02e3265341b8 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:35.579378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d396db-fd51-4d4b-b9e8-a0f7f67a6b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:35.581139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d396db-fd51-4d4b-b9e8-a0f7f67a6b37 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:35.581385 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d396db-fd51-4d4b-b9e8-a0f7f67a6b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:35.581781 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d396db-fd51-4d4b-b9e8-a0f7f67a6b37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:35.629730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:35.685831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d396db-fd51-4d4b-b9e8-a0f7f67a6b37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:35.687465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d396db-fd51-4d4b-b9e8-a0f7f67a6b37 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:35.821556 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 602/1214] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:07:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:07:35.827578 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43a820-6cd3-4f1c-a82a-2e15b3b555ca None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:35.851491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:35.957684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43a820-6cd3-4f1c-a82a-2e15b3b555ca None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:36.002169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bcc6ea-40b1-4424-aa93-a47608aa73cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:36.002887 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcc6ea-40b1-4424-aa93-a47608aa73cb None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:36.003268 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcc6ea-40b1-4424-aa93-a47608aa73cb None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:36.003538 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bcc6ea-40b1-4424-aa93-a47608aa73cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:36.015703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bcc6ea-40b1-4424-aa93-a47608aa73cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:36.017251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bcc6ea-40b1-4424-aa93-a47608aa73cb None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:36.034177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-b7bcc6ea-40b1-4424-aa93-a47608aa73cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:36.034177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:36.034177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:36.034177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:36.034177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:36.034177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:36.034177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:36.034177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:36.034177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:36.034177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:36.034177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:36.034177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:36.034177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:07:36.034177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:07:36.034177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:07:36.036269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:07:36.036269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:36.036269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:36.036269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:36.036269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:36.036269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:36.036269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:36.036269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:36.036269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:36.036269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:36.036269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:36.036269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:36.036269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:36.036269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:36.036269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:36.036269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:36.036269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:36.037588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:07:36.037588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:07:36.037588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:36.037588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:07:36.037588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 613/1215] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:07:36.043328 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d396db-fd51-4d4b-b9e8-a0f7f67a6b37 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:36.104990 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:36.150184 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d396db-fd51-4d4b-b9e8-a0f7f67a6b37 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:36.185721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc937716-9fb5-49fb-b138-2ecac54bfd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:36.186339 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc937716-9fb5-49fb-b138-2ecac54bfd0d None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:36.186613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc937716-9fb5-49fb-b138-2ecac54bfd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:36.186880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc937716-9fb5-49fb-b138-2ecac54bfd0d None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:36.187591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc937716-9fb5-49fb-b138-2ecac54bfd0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:36.187855 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc937716-9fb5-49fb-b138-2ecac54bfd0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:36.189240 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc937716-9fb5-49fb-b138-2ecac54bfd0d None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:36.205674 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 603/1216] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:07:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:36.211332 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bcc6ea-40b1-4424-aa93-a47608aa73cb None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:36.294802 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:36.354418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bcc6ea-40b1-4424-aa93-a47608aa73cb None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:36.393836 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4b483-09fc-4843-a7ae-9b440490316a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:36.394480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4b483-09fc-4843-a7ae-9b440490316a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:36.394749 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4b483-09fc-4843-a7ae-9b440490316a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:36.395020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd4b483-09fc-4843-a7ae-9b440490316a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:36.402999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4b483-09fc-4843-a7ae-9b440490316a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:36.404431 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd4b483-09fc-4843-a7ae-9b440490316a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:36.416840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-3dd4b483-09fc-4843-a7ae-9b440490316a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:07:36.416840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:36.416840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:36.416840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:36.416840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:36.416840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:36.416840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:36.416840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:36.416840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:36.416840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:36.416840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:36.416840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:36.416840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:07:36.416840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:07:36.416840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:07:36.417858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:07:36.417858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:36.417858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:36.417858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:36.417858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:36.417858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:36.417858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:36.417858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:36.417858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:36.417858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:36.417858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:36.417858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:36.417858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:36.417858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:36.417858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:36.417858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:36.417858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:36.418805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:07:36.418805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:07:36.418805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:07:36.418805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:07:36.418805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:07:36.418805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:07:36.418805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:07:36.418805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:07:36.418805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 614/1217] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:07:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:07:36.423845 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc937716-9fb5-49fb-b138-2ecac54bfd0d None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:36.534104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc937716-9fb5-49fb-b138-2ecac54bfd0d None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:36.559381 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:36.574907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48259aad-9319-48f3-be2b-2028f82f23ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:36.575612 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48259aad-9319-48f3-be2b-2028f82f23ec None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:36.575969 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48259aad-9319-48f3-be2b-2028f82f23ec None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:36.576312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48259aad-9319-48f3-be2b-2028f82f23ec None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:36.577337 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48259aad-9319-48f3-be2b-2028f82f23ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:36.577569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48259aad-9319-48f3-be2b-2028f82f23ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:36.579134 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48259aad-9319-48f3-be2b-2028f82f23ec None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:36.593509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 604/1218] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:07:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:36.600459 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4b483-09fc-4843-a7ae-9b440490316a None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:36.727164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd4b483-09fc-4843-a7ae-9b440490316a None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:36.761184 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:36.769101 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20824735-8272-4883-a80a-fc9c66729bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:36.769776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20824735-8272-4883-a80a-fc9c66729bef None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:36.770072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20824735-8272-4883-a80a-fc9c66729bef None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:36.770467 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20824735-8272-4883-a80a-fc9c66729bef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:36.771363 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20824735-8272-4883-a80a-fc9c66729bef None admin] RBAC: Adding query filter params (role.id=977824a8c6a24c00af2f6dabd0e2a6f0, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=796772cf491047c8b0a609dae9402307) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:36.771745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20824735-8272-4883-a80a-fc9c66729bef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:36.774899 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20824735-8272-4883-a80a-fc9c66729bef None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:36.788757 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 615/1219] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:07:36 2022] GET /identity/v3/role_assignments?user.id=796772cf491047c8b0a609dae9402307&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=977824a8c6a24c00af2f6dabd0e2a6f0 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:38.997864 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e17014-7570-42af-9169-e3b1ccfc6c4d None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:38.998457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e17014-7570-42af-9169-e3b1ccfc6c4d None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:38.998714 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e17014-7570-42af-9169-e3b1ccfc6c4d None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:39.001206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 605/1220] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:07:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:07:39.007968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5d208-cc32-4cfe-a309-a247caa6fa13 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:39.008486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5d208-cc32-4cfe-a309-a247caa6fa13 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:39.008563 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b5d208-cc32-4cfe-a309-a247caa6fa13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:39.027491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-02b5d208-cc32-4cfe-a309-a247caa6fa13 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:07:39.028713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-02b5d208-cc32-4cfe-a309-a247caa6fa13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:07:39.105701 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 616/1221] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:07:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:07:39.129065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e17014-7570-42af-9169-e3b1ccfc6c4d None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:39.233111 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e17014-7570-42af-9169-e3b1ccfc6c4d None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:39.268693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40491d-a3d2-4afb-b79f-71696a4ecd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:39.269323 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40491d-a3d2-4afb-b79f-71696a4ecd09 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:39.269568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40491d-a3d2-4afb-b79f-71696a4ecd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:39.269808 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e40491d-a3d2-4afb-b79f-71696a4ecd09 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:39.280006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e40491d-a3d2-4afb-b79f-71696a4ecd09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:07:39.285634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-6e40491d-a3d2-4afb-b79f-71696a4ecd09 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:07:39.285634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:39.285634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:39.285634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:39.285634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:39.285634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:39.285634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:39.285634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:39.285634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:39.285634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:39.285634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:39.285634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:39.285634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:07:39.285634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:07:39.285634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:07:39.285634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:07:39.286948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:07:39.286948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:07:39.286948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:07:39.286948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:07:39.286948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:39.286948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:39.286948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:39.286948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:39.286948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:39.286948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:39.286948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:39.286948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:39.286948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:39.286948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:39.286948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:39.286948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:39.288002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:39.288002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:39.288002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:39.288002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:39.288002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:07:39.288002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:07:39.288002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:07:39.288002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:07:39.288002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:07:39.288002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:07:39.288002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:07:39.288002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:07:39.288002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 606/1222] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:07:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:39.293092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5d208-cc32-4cfe-a309-a247caa6fa13 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:39.406742 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b5d208-cc32-4cfe-a309-a247caa6fa13 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:39.432305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:39.445405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8482f3-429c-4595-af5a-5832e7826c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:39.446037 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8482f3-429c-4595-af5a-5832e7826c59 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:39.446339 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8482f3-429c-4595-af5a-5832e7826c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:39.446692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8482f3-429c-4595-af5a-5832e7826c59 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:39.447424 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8482f3-429c-4595-af5a-5832e7826c59 None admin] RBAC: Adding query filter params (name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:39.447723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8482f3-429c-4595-af5a-5832e7826c59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:39.449118 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8482f3-429c-4595-af5a-5832e7826c59 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:39.457594 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 617/1223] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:07:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:39.464282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40491d-a3d2-4afb-b79f-71696a4ecd09 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:39.525363 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:39.586304 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e40491d-a3d2-4afb-b79f-71696a4ecd09 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:39.624677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae974c53-e5ed-4279-88f8-f8f169e156e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:39.625316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae974c53-e5ed-4279-88f8-f8f169e156e3 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:39.625591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae974c53-e5ed-4279-88f8-f8f169e156e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:39.625861 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae974c53-e5ed-4279-88f8-f8f169e156e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:39.636141 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae974c53-e5ed-4279-88f8-f8f169e156e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:39.637616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae974c53-e5ed-4279-88f8-f8f169e156e3 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:39.655193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-ae974c53-e5ed-4279-88f8-f8f169e156e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:39.655193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:39.655193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:39.655193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:39.655193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:39.655193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:39.655193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:39.655193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:39.655193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:39.655193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:39.655193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:39.655193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:39.655193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:07:39.655193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:07:39.655193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:07:39.656195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:07:39.656195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:39.656195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:39.656195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:39.656195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:39.656195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:39.656195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:39.656195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:39.656195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:39.656195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:39.656195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:39.656195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:39.656195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:39.656195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:39.656195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:39.656195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:39.656195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:39.657663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:07:39.657663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:07:39.657663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:39.657663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:07:39.660996 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 607/1224] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:39.663870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8482f3-429c-4595-af5a-5832e7826c59 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:39.773344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8482f3-429c-4595-af5a-5832e7826c59 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:39.807343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dd931-b6de-4313-81fd-7c92bd21ee6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:39.807879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dd931-b6de-4313-81fd-7c92bd21ee6a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:39.808158 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dd931-b6de-4313-81fd-7c92bd21ee6a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:39.808430 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dd931-b6de-4313-81fd-7c92bd21ee6a None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:39.809153 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dd931-b6de-4313-81fd-7c92bd21ee6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:39.809455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dd931-b6de-4313-81fd-7c92bd21ee6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:39.810839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dd931-b6de-4313-81fd-7c92bd21ee6a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:39.825092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 618/1225] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:39.831311 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae974c53-e5ed-4279-88f8-f8f169e156e3 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:39.942204 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:39.950189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae974c53-e5ed-4279-88f8-f8f169e156e3 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:40.001635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4fd04-d2b9-4c25-8608-47cb7d1389e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:40.003133 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4fd04-d2b9-4c25-8608-47cb7d1389e9 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:40.005037 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4fd04-d2b9-4c25-8608-47cb7d1389e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:40.005037 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4fd04-d2b9-4c25-8608-47cb7d1389e9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:40.012187 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4fd04-d2b9-4c25-8608-47cb7d1389e9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:40.013897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4fd04-d2b9-4c25-8608-47cb7d1389e9 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:40.027939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-83d4fd04-d2b9-4c25-8608-47cb7d1389e9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 02:07:40.027939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:40.027939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:40.027939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:40.027939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:40.027939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:40.027939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:40.027939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:40.027939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:40.027939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:40.027939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:40.027939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:40.027939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:07:40.027939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:07:40.027939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:07:40.027939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:40.029083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:40.030158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:40.030158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:40.030158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:07:40.030158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:07:40.030158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:07:40.030158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:07:40.030158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:07:40.030158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:07:40.030158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 02:07:40.030158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:07:40.030158 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 608/1226] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:07:40.035794 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dd931-b6de-4313-81fd-7c92bd21ee6a None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:40.088181 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:40.157196 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dd931-b6de-4313-81fd-7c92bd21ee6a None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:40.196142 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b32ac1-d509-471b-be9c-ecc6d22dd72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:40.196790 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b32ac1-d509-471b-be9c-ecc6d22dd72b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:40.197069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b32ac1-d509-471b-be9c-ecc6d22dd72b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:40.197418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b32ac1-d509-471b-be9c-ecc6d22dd72b None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:40.198151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b32ac1-d509-471b-be9c-ecc6d22dd72b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:40.198476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b32ac1-d509-471b-be9c-ecc6d22dd72b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:40.199908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b32ac1-d509-471b-be9c-ecc6d22dd72b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:40.217981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 619/1227] 10.222.0.45 () {56 vars in 1143 bytes} [Thu Aug 4 02:07:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:40.224226 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4fd04-d2b9-4c25-8608-47cb7d1389e9 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:40.344244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4fd04-d2b9-4c25-8608-47cb7d1389e9 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:40.382021 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5164bed-9537-4e17-a3cc-2ed033586e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:40.382921 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5164bed-9537-4e17-a3cc-2ed033586e19 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:40.383354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5164bed-9537-4e17-a3cc-2ed033586e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:40.383702 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5164bed-9537-4e17-a3cc-2ed033586e19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:40.397371 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5164bed-9537-4e17-a3cc-2ed033586e19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:40.399876 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5164bed-9537-4e17-a3cc-2ed033586e19 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:40.425388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-c5164bed-9537-4e17-a3cc-2ed033586e19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:40.425388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:40.425388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:40.425388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:40.425388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:40.425388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:40.425388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:40.425388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:40.425388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:40.425388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:40.425388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:40.425388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:40.425388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:07:40.425388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:07:40.425388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:07:40.426663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:07:40.426663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:40.426663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:40.426663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:40.426663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:40.426663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:40.426663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:40.426663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:40.426663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:40.426663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:40.426663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:40.426663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:40.426663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:40.426663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:40.426663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:40.426663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:40.426663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:40.427787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:07:40.427787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:07:40.427787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:40.427787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:07:40.427787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 609/1228] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:07:40.434792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b32ac1-d509-471b-be9c-ecc6d22dd72b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:40.549595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b32ac1-d509-471b-be9c-ecc6d22dd72b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:40.552514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:40.588664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cfd953-ad1d-4f22-aea0-cebd4109a836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:40.589352 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfd953-ad1d-4f22-aea0-cebd4109a836 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:40.589764 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfd953-ad1d-4f22-aea0-cebd4109a836 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:40.590181 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cfd953-ad1d-4f22-aea0-cebd4109a836 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:40.590878 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cfd953-ad1d-4f22-aea0-cebd4109a836 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:40.591186 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cfd953-ad1d-4f22-aea0-cebd4109a836 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:40.592813 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cfd953-ad1d-4f22-aea0-cebd4109a836 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:40.606201 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 620/1229] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:40.612829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5164bed-9537-4e17-a3cc-2ed033586e19 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:40.624747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:40.731505 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5164bed-9537-4e17-a3cc-2ed033586e19 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:40.768488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc056483-5afa-4d3e-b67c-3e780d8e0694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:40.769154 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc056483-5afa-4d3e-b67c-3e780d8e0694 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:40.769423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc056483-5afa-4d3e-b67c-3e780d8e0694 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:40.769697 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc056483-5afa-4d3e-b67c-3e780d8e0694 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:40.776662 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc056483-5afa-4d3e-b67c-3e780d8e0694 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:40.778150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc056483-5afa-4d3e-b67c-3e780d8e0694 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:40.789417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-dc056483-5afa-4d3e-b67c-3e780d8e0694 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:07:40.789417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:40.789417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:40.789417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:40.789417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:40.789417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:40.789417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:40.789417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:40.789417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:40.789417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:40.789417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:40.789417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:40.789417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:07:40.789417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:07:40.789417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:07:40.790455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:07:40.790455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:40.790455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:40.790455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:40.790455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:40.790455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:40.790455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:40.790455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:40.790455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:40.790455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:40.790455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:40.790455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:40.790455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:40.790455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:40.790455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:40.790455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:40.790455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:40.791399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:07:40.791399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:07:40.791399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:07:40.791399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:07:40.791399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:07:40.791399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:07:40.791399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:07:40.791399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:07:40.791399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 610/1230] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:07:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:40.796576 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cfd953-ad1d-4f22-aea0-cebd4109a836 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:40.908888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cfd953-ad1d-4f22-aea0-cebd4109a836 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:40.945269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504237e7-3e97-4623-9541-88769d7d2a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:40.945891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504237e7-3e97-4623-9541-88769d7d2a68 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:40.946171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504237e7-3e97-4623-9541-88769d7d2a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:40.946446 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504237e7-3e97-4623-9541-88769d7d2a68 None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:40.947362 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504237e7-3e97-4623-9541-88769d7d2a68 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:40.947677 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504237e7-3e97-4623-9541-88769d7d2a68 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:40.949273 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504237e7-3e97-4623-9541-88769d7d2a68 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:40.963976 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 621/1231] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:07:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:40.970465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc056483-5afa-4d3e-b67c-3e780d8e0694 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:41.047186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:41.091752 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc056483-5afa-4d3e-b67c-3e780d8e0694 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:41.131610 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d95f6-55b1-44d0-a6fb-0985006a4123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:41.132283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d95f6-55b1-44d0-a6fb-0985006a4123 None admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:41.132563 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d95f6-55b1-44d0-a6fb-0985006a4123 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:41.132869 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d95f6-55b1-44d0-a6fb-0985006a4123 None admin] PATH_INFO: `/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/796772cf491047c8b0a609dae9402307/roles/977824a8c6a24c00af2f6dabd0e2a6f0` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:41.148584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d95f6-55b1-44d0-a6fb-0985006a4123 None admin] RBAC: Authorizing `identity:create_grant(project_id=74b8caff828a4f5289094fb2ae2278f7, user_id=796772cf491047c8b0a609dae9402307, role_id=977824a8c6a24c00af2f6dabd0e2a6f0)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:41.150416 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d95f6-55b1-44d0-a6fb-0985006a4123 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:41.164279 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 611/1232] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:07:40 2022] PUT /identity/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/796772cf491047c8b0a609dae9402307/roles/977824a8c6a24c00af2f6dabd0e2a6f0 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:07:43.382051 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15956be7-15e9-4e54-847f-000fa38398c2 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:43.382688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15956be7-15e9-4e54-847f-000fa38398c2 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:43.382688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15956be7-15e9-4e54-847f-000fa38398c2 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:43.383946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 622/1233] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:07:43.389206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dee5e-b105-4c9b-b193-d1d5399d4edb None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:43.389957 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dee5e-b105-4c9b-b193-d1d5399d4edb None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:43.390128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83dee5e-b105-4c9b-b193-d1d5399d4edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:43.408523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-a83dee5e-b105-4c9b-b193-d1d5399d4edb None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:07:43.410893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a83dee5e-b105-4c9b-b193-d1d5399d4edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:07:43.480217 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 612/1234] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:07:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:07:43.504479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15956be7-15e9-4e54-847f-000fa38398c2 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:43.541224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:43.645469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15956be7-15e9-4e54-847f-000fa38398c2 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:43.694059 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07df2a-7639-419f-8ffa-5dcdc08827e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:43.694701 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07df2a-7639-419f-8ffa-5dcdc08827e2 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:43.694980 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07df2a-7639-419f-8ffa-5dcdc08827e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:43.695254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07df2a-7639-419f-8ffa-5dcdc08827e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:43.707181 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07df2a-7639-419f-8ffa-5dcdc08827e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:07:43.713378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-4b07df2a-7639-419f-8ffa-5dcdc08827e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:07:43.713378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:43.713378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:43.713378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:43.713378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:43.713378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:43.713378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:43.713378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:43.713378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:43.713378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:43.713378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:43.713378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:43.713378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:07:43.713378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:07:43.713378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:07:43.713378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:07:43.714651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:07:43.714651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:07:43.714651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:07:43.714651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:07:43.714651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:43.714651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:43.714651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:43.714651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:43.714651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:43.714651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:43.714651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:43.714651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:43.714651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:43.714651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:43.714651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:43.714651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:43.715880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:43.715880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:43.715880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:43.715880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:43.715880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:07:43.715880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:07:43.715880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:07:43.715880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:07:43.715880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:07:43.715880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:07:43.715880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 04 02:07:43.715880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:07:43.721418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 623/1235] 10.222.0.45 () {56 vars in 1102 bytes} [Thu Aug 4 02:07:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:43.722503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dee5e-b105-4c9b-b193-d1d5399d4edb None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:43.772982 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:43.837398 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83dee5e-b105-4c9b-b193-d1d5399d4edb None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:43.878188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ffad7-f75a-4dd0-a29b-816ee7e973db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:43.878857 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ffad7-f75a-4dd0-a29b-816ee7e973db None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:43.879185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ffad7-f75a-4dd0-a29b-816ee7e973db None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:43.879463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ffad7-f75a-4dd0-a29b-816ee7e973db None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:43.880297 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ffad7-f75a-4dd0-a29b-816ee7e973db None admin] RBAC: Adding query filter params (name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:43.880597 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ffad7-f75a-4dd0-a29b-816ee7e973db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:43.882082 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ffad7-f75a-4dd0-a29b-816ee7e973db None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:43.892280 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 613/1236] 10.222.0.45 () {56 vars in 1103 bytes} [Thu Aug 4 02:07:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:43.898848 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07df2a-7639-419f-8ffa-5dcdc08827e2 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:43.999135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:44.015277 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07df2a-7639-419f-8ffa-5dcdc08827e2 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:44.054410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636547c-b7b5-4557-afec-37102edd1212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:44.055174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636547c-b7b5-4557-afec-37102edd1212 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:44.055482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636547c-b7b5-4557-afec-37102edd1212 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:44.055815 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636547c-b7b5-4557-afec-37102edd1212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:44.066925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636547c-b7b5-4557-afec-37102edd1212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:44.068648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636547c-b7b5-4557-afec-37102edd1212 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:44.085344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-9636547c-b7b5-4557-afec-37102edd1212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:44.085344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:44.085344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:44.085344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:44.085344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:44.085344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:44.085344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:44.085344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:44.085344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:44.085344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:44.085344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:44.085344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:44.085344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:07:44.085344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:07:44.085344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:07:44.086518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:07:44.086518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:44.086518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:44.086518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:44.086518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:44.086518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:44.086518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:44.086518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:44.086518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:44.086518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:44.086518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:44.086518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:44.086518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:44.086518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:44.086518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:44.086518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:44.086518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:44.087482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:07:44.087482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:07:44.087482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:44.087482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:07:44.087482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 624/1237] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:44.092742 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ffad7-f75a-4dd0-a29b-816ee7e973db None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:44.204743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ffad7-f75a-4dd0-a29b-816ee7e973db None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:44.215394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:44.243829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72263f-ef8e-4d5e-843c-69a69c66b896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:44.244589 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72263f-ef8e-4d5e-843c-69a69c66b896 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:44.244903 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72263f-ef8e-4d5e-843c-69a69c66b896 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:44.245175 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72263f-ef8e-4d5e-843c-69a69c66b896 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:44.245866 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72263f-ef8e-4d5e-843c-69a69c66b896 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:44.246314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72263f-ef8e-4d5e-843c-69a69c66b896 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:44.247957 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72263f-ef8e-4d5e-843c-69a69c66b896 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:44.268268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 614/1238] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:44.269814 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636547c-b7b5-4557-afec-37102edd1212 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:44.397110 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636547c-b7b5-4557-afec-37102edd1212 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:44.440687 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b0f46-c868-4dc4-8e2e-94db48d451df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:44.441535 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b0f46-c868-4dc4-8e2e-94db48d451df None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:44.441926 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b0f46-c868-4dc4-8e2e-94db48d451df None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:44.442309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b0f46-c868-4dc4-8e2e-94db48d451df None admin] PATH_INFO: `/v3/users/placement` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:44.447075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:44.457104 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b0f46-c868-4dc4-8e2e-94db48d451df None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:44.458748 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b0f46-c868-4dc4-8e2e-94db48d451df None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:44.473664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-d49b0f46-c868-4dc4-8e2e-94db48d451df None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 02:07:44.473664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:44.473664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:44.473664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:44.473664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:44.473664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:44.473664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:44.473664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:44.473664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:44.473664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:44.473664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:44.473664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:44.473664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:07:44.473664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:07:44.473664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:07:44.473664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:44.474855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:44.476077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:44.476077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:44.476077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:07:44.476077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:07:44.476077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:07:44.476077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:07:44.476077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:07:44.476077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:07:44.476077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 02:07:44.476077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:07:44.476077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 625/1239] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 04 02:07:44.481220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72263f-ef8e-4d5e-843c-69a69c66b896 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:44.595325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72263f-ef8e-4d5e-843c-69a69c66b896 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:44.632775 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb02fc-8103-4be6-ab9d-3388b47177f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:44.633382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb02fc-8103-4be6-ab9d-3388b47177f5 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:44.633673 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb02fc-8103-4be6-ab9d-3388b47177f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:44.634098 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb02fc-8103-4be6-ab9d-3388b47177f5 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:44.635128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb02fc-8103-4be6-ab9d-3388b47177f5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:44.635503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb02fc-8103-4be6-ab9d-3388b47177f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:44.637163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb02fc-8103-4be6-ab9d-3388b47177f5 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:44.660874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 615/1240] 10.222.0.45 () {56 vars in 1143 bytes} [Thu Aug 4 02:07:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:44.667287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b0f46-c868-4dc4-8e2e-94db48d451df None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:44.795027 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b0f46-c868-4dc4-8e2e-94db48d451df None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:44.836226 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10a2af-261e-4873-8c41-aa729f77eb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:44.837006 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10a2af-261e-4873-8c41-aa729f77eb49 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:44.837338 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10a2af-261e-4873-8c41-aa729f77eb49 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:44.837663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10a2af-261e-4873-8c41-aa729f77eb49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:44.907075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:44.915323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee10a2af-261e-4873-8c41-aa729f77eb49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:44.916951 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee10a2af-261e-4873-8c41-aa729f77eb49 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:45.021089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 626/1241] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:07:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:07:45.027059 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb02fc-8103-4be6-ab9d-3388b47177f5 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:45.056758 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:45.138623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb02fc-8103-4be6-ab9d-3388b47177f5 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:45.178333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293dc77-561c-4d41-9351-faa34c3fdd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:45.178333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293dc77-561c-4d41-9351-faa34c3fdd62 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:45.178333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293dc77-561c-4d41-9351-faa34c3fdd62 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:45.178333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293dc77-561c-4d41-9351-faa34c3fdd62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:45.190618 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293dc77-561c-4d41-9351-faa34c3fdd62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:45.192270 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293dc77-561c-4d41-9351-faa34c3fdd62 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:45.213627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-f293dc77-561c-4d41-9351-faa34c3fdd62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:45.213627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:45.213627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:45.213627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:45.213627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:45.213627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:45.213627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:45.213627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:45.213627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:45.213627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:45.213627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:45.213627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:45.213627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:07:45.213627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:07:45.213627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:07:45.214880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:07:45.214880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:45.214880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:45.214880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:45.214880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:45.214880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:45.214880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:45.214880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:45.214880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:45.214880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:45.214880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:45.214880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:45.214880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:45.214880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:45.214880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:45.214880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:45.214880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:45.220570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:07:45.220570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:07:45.220570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:45.220570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:07:45.220570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 616/1242] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:45.222091 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10a2af-261e-4873-8c41-aa729f77eb49 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:45.363434 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:45.370826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10a2af-261e-4873-8c41-aa729f77eb49 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:45.412831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b7552-cdc0-4dee-a701-0f6a7356ee76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:45.413584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b7552-cdc0-4dee-a701-0f6a7356ee76 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:45.413829 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b7552-cdc0-4dee-a701-0f6a7356ee76 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:45.414099 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b7552-cdc0-4dee-a701-0f6a7356ee76 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:45.414939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b7552-cdc0-4dee-a701-0f6a7356ee76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:45.415246 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b7552-cdc0-4dee-a701-0f6a7356ee76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:45.416798 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b7552-cdc0-4dee-a701-0f6a7356ee76 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:45.431360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 627/1243] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:45.437180 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293dc77-561c-4d41-9351-faa34c3fdd62 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:45.524012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:45.547244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293dc77-561c-4d41-9351-faa34c3fdd62 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:45.584576 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912b689-8fe2-4d41-86dc-a345a03ad951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:45.585244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912b689-8fe2-4d41-86dc-a345a03ad951 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:45.585526 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912b689-8fe2-4d41-86dc-a345a03ad951 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:45.585802 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912b689-8fe2-4d41-86dc-a345a03ad951 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:45.592453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912b689-8fe2-4d41-86dc-a345a03ad951 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:45.593929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912b689-8fe2-4d41-86dc-a345a03ad951 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:45.606599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-c912b689-8fe2-4d41-86dc-a345a03ad951 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:07:45.606599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:45.606599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:45.606599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:45.606599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:45.606599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:45.606599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:45.606599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:45.606599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:45.606599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:45.606599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:45.606599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:45.606599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:07:45.606599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:07:45.606599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:07:45.607889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:07:45.607889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:45.607889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:45.607889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:45.607889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:45.607889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:45.607889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:45.607889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:45.607889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:45.607889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:45.607889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:45.607889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:45.607889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:45.607889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:45.607889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:45.607889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:45.607889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:45.609060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:07:45.609060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:07:45.609060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:07:45.609060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:07:45.609060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:07:45.609060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:07:45.609060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:07:45.609060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:07:45.609060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 617/1244] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:07:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:45.614958 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b7552-cdc0-4dee-a701-0f6a7356ee76 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:45.734695 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b7552-cdc0-4dee-a701-0f6a7356ee76 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:45.773783 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ce8b8-6a2b-49c5-af3f-90bdaf5ce9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:45.774379 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ce8b8-6a2b-49c5-af3f-90bdaf5ce9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:45.774729 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ce8b8-6a2b-49c5-af3f-90bdaf5ce9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:45.775030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ce8b8-6a2b-49c5-af3f-90bdaf5ce9e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:45.775817 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ce8b8-6a2b-49c5-af3f-90bdaf5ce9e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:45.776144 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ce8b8-6a2b-49c5-af3f-90bdaf5ce9e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:45.777693 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ce8b8-6a2b-49c5-af3f-90bdaf5ce9e5 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:45.791016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 628/1245] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:07:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:45.798000 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912b689-8fe2-4d41-86dc-a345a03ad951 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:45.916307 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912b689-8fe2-4d41-86dc-a345a03ad951 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:45.952103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5977f59-1dcc-4b69-a74d-d4123a206d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:45.952773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5977f59-1dcc-4b69-a74d-d4123a206d6a None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:45.953050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5977f59-1dcc-4b69-a74d-d4123a206d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:45.953324 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5977f59-1dcc-4b69-a74d-d4123a206d6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:45.954104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5977f59-1dcc-4b69-a74d-d4123a206d6a None admin] RBAC: Adding query filter params (role.id=977824a8c6a24c00af2f6dabd0e2a6f0, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=796772cf491047c8b0a609dae9402307) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:45.954413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5977f59-1dcc-4b69-a74d-d4123a206d6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:45.955886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5977f59-1dcc-4b69-a74d-d4123a206d6a None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:45.962197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 618/1246] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:07:45 2022] GET /identity/v3/role_assignments?user.id=796772cf491047c8b0a609dae9402307&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=977824a8c6a24c00af2f6dabd0e2a6f0 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:48.144522 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39811be-e04f-4b50-a610-e4abb3a5c1bd None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:48.145097 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39811be-e04f-4b50-a610-e4abb3a5c1bd None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:48.145097 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39811be-e04f-4b50-a610-e4abb3a5c1bd None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:48.146449 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 629/1247] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:07:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:07:48.151949 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb1e90-8b10-4455-855c-088a3a0f5d2a None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:48.152508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb1e90-8b10-4455-855c-088a3a0f5d2a None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:48.152508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fb1e90-8b10-4455-855c-088a3a0f5d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:48.159777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:48.175786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-a4fb1e90-8b10-4455-855c-088a3a0f5d2a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:07:48.177024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a4fb1e90-8b10-4455-855c-088a3a0f5d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:07:48.255416 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 619/1248] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:07:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:07:48.279669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39811be-e04f-4b50-a610-e4abb3a5c1bd None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:48.302885 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:48.420805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39811be-e04f-4b50-a610-e4abb3a5c1bd None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:48.466321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a7e84-a75e-41db-92e4-8ee5046d0d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:48.467391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a7e84-a75e-41db-92e4-8ee5046d0d84 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:48.467837 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a7e84-a75e-41db-92e4-8ee5046d0d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:48.468424 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a7e84-a75e-41db-92e4-8ee5046d0d84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:48.480758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a7e84-a75e-41db-92e4-8ee5046d0d84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:07:48.486640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-419a7e84-a75e-41db-92e4-8ee5046d0d84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:07:48.486640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:48.486640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:48.486640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:48.486640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:48.486640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:48.486640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:48.486640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:48.486640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:48.486640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:48.486640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:48.486640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:48.486640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:07:48.486640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:07:48.486640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:07:48.486640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:07:48.488712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:07:48.488712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:07:48.488712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:07:48.488712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:07:48.488712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:48.488712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:48.488712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:48.488712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:48.488712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:48.488712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:48.488712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:48.488712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:48.488712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:48.488712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:48.488712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:48.488712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:48.490638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:48.490638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:48.490638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:48.490638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:48.490638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:07:48.490638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:07:48.490638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:07:48.490638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:07:48.490638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:07:48.490638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:07:48.490638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:07:48.490638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:07:48.490638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 630/1249] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:07:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:48.496842 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb1e90-8b10-4455-855c-088a3a0f5d2a None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:48.603919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fb1e90-8b10-4455-855c-088a3a0f5d2a None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:48.638874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:48.647214 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19490a3e-4f46-4721-820e-2cf6b6ef657f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:48.647951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19490a3e-4f46-4721-820e-2cf6b6ef657f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:48.648331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19490a3e-4f46-4721-820e-2cf6b6ef657f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:48.648675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19490a3e-4f46-4721-820e-2cf6b6ef657f None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:48.649516 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19490a3e-4f46-4721-820e-2cf6b6ef657f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:48.649819 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19490a3e-4f46-4721-820e-2cf6b6ef657f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:48.651214 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19490a3e-4f46-4721-820e-2cf6b6ef657f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:48.660979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 620/1250] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:07:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:07:48.667125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a7e84-a75e-41db-92e4-8ee5046d0d84 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:48.757215 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:48.784680 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a7e84-a75e-41db-92e4-8ee5046d0d84 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:48.824567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0acc331-8e97-43a6-868e-618f4d9aab1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:48.825380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0acc331-8e97-43a6-868e-618f4d9aab1a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:48.825760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0acc331-8e97-43a6-868e-618f4d9aab1a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:48.826132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0acc331-8e97-43a6-868e-618f4d9aab1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:48.836712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0acc331-8e97-43a6-868e-618f4d9aab1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:48.838264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0acc331-8e97-43a6-868e-618f4d9aab1a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:48.854864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-b0acc331-8e97-43a6-868e-618f4d9aab1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:48.854864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:48.854864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:48.854864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:48.854864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:48.854864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:48.854864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:48.854864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:48.854864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:48.854864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:48.854864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:48.854864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:48.854864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:07:48.854864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:07:48.854864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:07:48.856005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:07:48.856005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:48.856005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:48.856005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:48.856005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:48.856005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:48.856005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:48.856005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:48.856005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:48.856005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:48.856005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:48.856005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:48.856005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:48.856005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:48.856005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:48.856005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:48.856005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:48.857152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:07:48.857152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:07:48.857152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:48.857152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:07:48.857152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 631/1251] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:48.865035 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19490a3e-4f46-4721-820e-2cf6b6ef657f None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:48.995991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19490a3e-4f46-4721-820e-2cf6b6ef657f None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:49.039461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abea76b-aac8-48f7-8617-f6950ce08536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:49.039942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abea76b-aac8-48f7-8617-f6950ce08536 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:49.040269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abea76b-aac8-48f7-8617-f6950ce08536 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:49.040533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abea76b-aac8-48f7-8617-f6950ce08536 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:49.041377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abea76b-aac8-48f7-8617-f6950ce08536 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:49.041728 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abea76b-aac8-48f7-8617-f6950ce08536 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:49.043375 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abea76b-aac8-48f7-8617-f6950ce08536 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:49.057715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 621/1252] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:49.063829 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0acc331-8e97-43a6-868e-618f4d9aab1a None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:49.183687 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0acc331-8e97-43a6-868e-618f4d9aab1a None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:49.213149 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:49.222640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc998f-8303-4b88-b361-fd003e364d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:49.223262 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc998f-8303-4b88-b361-fd003e364d0b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:49.223549 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc998f-8303-4b88-b361-fd003e364d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:49.223823 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcc998f-8303-4b88-b361-fd003e364d0b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:49.233235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc998f-8303-4b88-b361-fd003e364d0b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:49.234632 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcc998f-8303-4b88-b361-fd003e364d0b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:49.246816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-cfcc998f-8303-4b88-b361-fd003e364d0b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 02:07:49.246816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:49.246816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:49.246816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:49.246816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:49.246816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:49.246816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:49.246816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:49.246816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:49.246816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:49.246816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:49.246816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:49.246816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:07:49.246816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:07:49.246816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:07:49.246816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:49.248004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:49.249234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:49.249234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:49.249234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:07:49.249234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:07:49.249234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:07:49.249234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:07:49.249234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:07:49.249234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:07:49.249234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 02:07:49.249234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:07:49.252819 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 632/1253] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:49.253986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abea76b-aac8-48f7-8617-f6950ce08536 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:49.317443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:49.380646 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abea76b-aac8-48f7-8617-f6950ce08536 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:49.419917 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7b4a4-a281-4b64-b78d-1dd2044d1f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:49.420545 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7b4a4-a281-4b64-b78d-1dd2044d1f83 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:49.420869 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7b4a4-a281-4b64-b78d-1dd2044d1f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:49.421125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc7b4a4-a281-4b64-b78d-1dd2044d1f83 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:49.421942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc7b4a4-a281-4b64-b78d-1dd2044d1f83 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:49.422246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc7b4a4-a281-4b64-b78d-1dd2044d1f83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:49.423674 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc7b4a4-a281-4b64-b78d-1dd2044d1f83 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:49.448041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 622/1254] 10.222.0.45 () {56 vars in 1143 bytes} [Thu Aug 4 02:07:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:49.449406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc998f-8303-4b88-b361-fd003e364d0b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:49.568678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcc998f-8303-4b88-b361-fd003e364d0b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:49.604868 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06129f7b-f88e-4ec4-a0b7-c8af582fd097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:49.605482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06129f7b-f88e-4ec4-a0b7-c8af582fd097 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:49.605760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06129f7b-f88e-4ec4-a0b7-c8af582fd097 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:49.606029 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06129f7b-f88e-4ec4-a0b7-c8af582fd097 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:49.676032 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:49.691831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06129f7b-f88e-4ec4-a0b7-c8af582fd097 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:49.694671 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06129f7b-f88e-4ec4-a0b7-c8af582fd097 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:49.813208 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 633/1255] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:07:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 04 02:07:49.820261 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7b4a4-a281-4b64-b78d-1dd2044d1f83 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:49.931009 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc7b4a4-a281-4b64-b78d-1dd2044d1f83 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:49.946309 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:49.976562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadaa705-8198-4593-8aa9-1e65579e4a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:49.977224 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaa705-8198-4593-8aa9-1e65579e4a59 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:49.977558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaa705-8198-4593-8aa9-1e65579e4a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:49.977841 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaa705-8198-4593-8aa9-1e65579e4a59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:49.989971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadaa705-8198-4593-8aa9-1e65579e4a59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:49.991456 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadaa705-8198-4593-8aa9-1e65579e4a59 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:50.007621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-dadaa705-8198-4593-8aa9-1e65579e4a59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:50.007621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:50.007621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:50.007621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:50.007621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:50.007621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:50.007621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:50.007621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:50.007621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:50.007621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:50.007621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:50.007621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:50.007621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:07:50.007621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:07:50.007621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:07:50.008920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:07:50.008920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:50.008920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:50.008920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:50.008920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:50.008920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:50.008920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:50.008920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:50.008920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:50.008920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:50.008920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:50.008920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:50.008920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:50.008920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:50.008920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:50.008920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:50.008920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:50.010072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:07:50.010072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:07:50.010072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:50.010072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:07:50.010072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 623/1256] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:50.016020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06129f7b-f88e-4ec4-a0b7-c8af582fd097 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:50.110327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:50.127969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06129f7b-f88e-4ec4-a0b7-c8af582fd097 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:50.164612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e8646-2ef4-4989-b36d-3387416b4321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:50.165241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e8646-2ef4-4989-b36d-3387416b4321 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:50.165518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e8646-2ef4-4989-b36d-3387416b4321 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:50.165867 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e8646-2ef4-4989-b36d-3387416b4321 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:50.166661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e8646-2ef4-4989-b36d-3387416b4321 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:50.166955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e8646-2ef4-4989-b36d-3387416b4321 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:50.168357 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e8646-2ef4-4989-b36d-3387416b4321 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:50.186090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 634/1257] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:50.193299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadaa705-8198-4593-8aa9-1e65579e4a59 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:50.302383 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadaa705-8198-4593-8aa9-1e65579e4a59 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:50.343412 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b4c5b-fc62-47ba-9cc1-768d4c77d4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:50.344062 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b4c5b-fc62-47ba-9cc1-768d4c77d4df None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:50.344348 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b4c5b-fc62-47ba-9cc1-768d4c77d4df None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:50.344609 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7b4c5b-fc62-47ba-9cc1-768d4c77d4df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:50.352271 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b4c5b-fc62-47ba-9cc1-768d4c77d4df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:50.353763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7b4c5b-fc62-47ba-9cc1-768d4c77d4df None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:50.366461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-5a7b4c5b-fc62-47ba-9cc1-768d4c77d4df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:07:50.366461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:50.366461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:50.366461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:50.366461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:50.366461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:50.366461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:50.366461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:50.366461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:50.366461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:50.366461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:50.366461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:50.366461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:07:50.366461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:07:50.366461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:07:50.367649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:07:50.367649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:50.367649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:50.367649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:50.367649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:50.367649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:50.367649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:50.367649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:50.367649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:50.367649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:50.367649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:50.367649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:50.367649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:50.367649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:50.367649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:50.367649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:50.367649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:50.368748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:07:50.368748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:07:50.368748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:07:50.368748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:07:50.368748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:07:50.368748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:07:50.368748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:07:50.368748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:07:50.368748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 624/1258] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:07:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:50.374816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e8646-2ef4-4989-b36d-3387416b4321 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:50.486325 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e8646-2ef4-4989-b36d-3387416b4321 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:50.523128 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5dc128-bdb3-4d3d-9396-c60bc1a76184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:50.524005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dc128-bdb3-4d3d-9396-c60bc1a76184 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:50.524415 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dc128-bdb3-4d3d-9396-c60bc1a76184 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:50.524726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5dc128-bdb3-4d3d-9396-c60bc1a76184 None admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:50.525636 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5dc128-bdb3-4d3d-9396-c60bc1a76184 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:50.525941 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5dc128-bdb3-4d3d-9396-c60bc1a76184 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:50.527476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5dc128-bdb3-4d3d-9396-c60bc1a76184 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:50.540964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 635/1259] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:07:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:50.547568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b4c5b-fc62-47ba-9cc1-768d4c77d4df None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:50.576689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:50.657161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7b4c5b-fc62-47ba-9cc1-768d4c77d4df None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:50.694857 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b9d668-92d4-4694-9640-97bd1cc954b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:50.695443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9d668-92d4-4694-9640-97bd1cc954b7 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:50.695730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9d668-92d4-4694-9640-97bd1cc954b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:50.695987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9d668-92d4-4694-9640-97bd1cc954b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:50.696810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9d668-92d4-4694-9640-97bd1cc954b7 None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=796772cf491047c8b0a609dae9402307) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:50.697109 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9d668-92d4-4694-9640-97bd1cc954b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:50.698545 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b9d668-92d4-4694-9640-97bd1cc954b7 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:50.704368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 625/1260] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:07:50 2022] GET /identity/v3/role_assignments?user.id=796772cf491047c8b0a609dae9402307&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=35205394c8af4c52a61b12f92ba16644 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:52.729135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4fe9a-0806-47d6-b22a-2e1ce56e6579 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:52.729668 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4fe9a-0806-47d6-b22a-2e1ce56e6579 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:52.729668 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4fe9a-0806-47d6-b22a-2e1ce56e6579 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:52.731020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 636/1261] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:07:52.736031 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568ab06-fb63-4217-8bf7-907aaf75e92a None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:52.736423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568ab06-fb63-4217-8bf7-907aaf75e92a None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:52.736605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568ab06-fb63-4217-8bf7-907aaf75e92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:52.754613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-1568ab06-fb63-4217-8bf7-907aaf75e92a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:07:52.755798 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-1568ab06-fb63-4217-8bf7-907aaf75e92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:07:52.819619 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 626/1262] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:07:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:07:52.842445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4fe9a-0806-47d6-b22a-2e1ce56e6579 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:52.853337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:52.943692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4fe9a-0806-47d6-b22a-2e1ce56e6579 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:52.977969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ea959-7a02-470a-9590-99427f17a97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:52.978597 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ea959-7a02-470a-9590-99427f17a97b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:52.978879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ea959-7a02-470a-9590-99427f17a97b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:52.979146 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ea959-7a02-470a-9590-99427f17a97b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:52.989068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ea959-7a02-470a-9590-99427f17a97b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:07:52.993886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-067ea959-7a02-470a-9590-99427f17a97b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:07:52.993886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:52.993886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:52.993886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:52.993886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:52.993886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:52.993886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:52.993886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:52.993886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:52.993886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:52.993886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:52.993886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:52.993886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:07:52.993886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:07:52.993886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:07:52.993886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:07:52.994999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:07:52.994999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:07:52.994999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:07:52.994999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:07:52.994999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:52.994999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:52.994999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:52.994999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:52.994999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:52.994999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:52.994999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:52.994999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:52.994999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:52.994999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:52.994999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:52.994999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:52.996140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:52.996140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:52.996140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:52.996140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:52.996140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:07:52.996140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:07:52.996140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:07:52.996140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:07:52.996140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:07:52.996140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:07:52.996140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:07:52.996140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:07:52.996140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 637/1263] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:07:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:53.001088 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568ab06-fb63-4217-8bf7-907aaf75e92a None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:53.024535 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:53.102838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568ab06-fb63-4217-8bf7-907aaf75e92a None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:53.135947 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89032d2b-771d-40fe-9949-7931ed3d8500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:53.136556 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89032d2b-771d-40fe-9949-7931ed3d8500 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:53.136933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89032d2b-771d-40fe-9949-7931ed3d8500 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:53.137209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89032d2b-771d-40fe-9949-7931ed3d8500 None admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:53.137917 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89032d2b-771d-40fe-9949-7931ed3d8500 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:53.138221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89032d2b-771d-40fe-9949-7931ed3d8500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:53.139588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89032d2b-771d-40fe-9949-7931ed3d8500 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:53.147414 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 627/1264] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:07:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:53.153237 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ea959-7a02-470a-9590-99427f17a97b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:53.218583 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:53.251878 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ea959-7a02-470a-9590-99427f17a97b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:53.284559 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80935480-7eaf-47ae-ba22-971900eac8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:53.285212 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80935480-7eaf-47ae-ba22-971900eac8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:53.285493 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80935480-7eaf-47ae-ba22-971900eac8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:53.285767 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80935480-7eaf-47ae-ba22-971900eac8d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:53.296806 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80935480-7eaf-47ae-ba22-971900eac8d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:53.298564 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80935480-7eaf-47ae-ba22-971900eac8d0 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:53.317726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-80935480-7eaf-47ae-ba22-971900eac8d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:53.317726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:53.317726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:53.317726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:53.317726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:53.317726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:53.317726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:53.317726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:53.317726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:53.317726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:53.317726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:53.317726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:53.317726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:07:53.317726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:07:53.317726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:07:53.319066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:07:53.319066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:53.319066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:53.319066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:53.319066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:53.319066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:53.319066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:53.319066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:53.319066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:53.319066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:53.319066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:53.319066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:53.319066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:53.319066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:53.319066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:53.319066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:53.319066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:53.320072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:07:53.320072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:07:53.320072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:53.320072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:07:53.320072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 638/1265] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:53.325900 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89032d2b-771d-40fe-9949-7931ed3d8500 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:53.415078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:53.434339 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89032d2b-771d-40fe-9949-7931ed3d8500 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:53.466570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a5200-47eb-4c08-b1af-108ce0e3ddd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:53.467176 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a5200-47eb-4c08-b1af-108ce0e3ddd8 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:53.467456 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a5200-47eb-4c08-b1af-108ce0e3ddd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:53.467730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a5200-47eb-4c08-b1af-108ce0e3ddd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:53.468410 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a5200-47eb-4c08-b1af-108ce0e3ddd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:53.468794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a5200-47eb-4c08-b1af-108ce0e3ddd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:53.470106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8a5200-47eb-4c08-b1af-108ce0e3ddd8 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:53.482692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 628/1266] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:07:53.489031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80935480-7eaf-47ae-ba22-971900eac8d0 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:53.586875 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80935480-7eaf-47ae-ba22-971900eac8d0 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:53.606225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:53.620639 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cece8-4e96-4d66-84a9-b8bfc49b707c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:53.621283 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cece8-4e96-4d66-84a9-b8bfc49b707c None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:53.621565 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cece8-4e96-4d66-84a9-b8bfc49b707c None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:53.621837 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cece8-4e96-4d66-84a9-b8bfc49b707c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:53.629103 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1cece8-4e96-4d66-84a9-b8bfc49b707c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:53.630522 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1cece8-4e96-4d66-84a9-b8bfc49b707c None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:53.641443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-5b1cece8-4e96-4d66-84a9-b8bfc49b707c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 02:07:53.641443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:53.641443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:53.641443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:53.641443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:53.641443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:53.641443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:53.641443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:53.641443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:53.641443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:53.641443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:53.641443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:53.641443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:07:53.641443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:07:53.641443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:07:53.641443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:53.642561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:53.643884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:53.643884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:53.643884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:07:53.643884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:07:53.643884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:07:53.643884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:07:53.643884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:07:53.643884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:07:53.643884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 02:07:53.643884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:07:53.643884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 639/1267] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:53.648492 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a5200-47eb-4c08-b1af-108ce0e3ddd8 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:53.744522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a5200-47eb-4c08-b1af-108ce0e3ddd8 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:53.782965 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8860703-3a84-4944-a91e-a8b1f42b1c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:53.784091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8860703-3a84-4944-a91e-a8b1f42b1c22 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:53.784368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8860703-3a84-4944-a91e-a8b1f42b1c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:53.784655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8860703-3a84-4944-a91e-a8b1f42b1c22 None admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:53.785454 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8860703-3a84-4944-a91e-a8b1f42b1c22 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:53.785753 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8860703-3a84-4944-a91e-a8b1f42b1c22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:53.787175 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8860703-3a84-4944-a91e-a8b1f42b1c22 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:53.795751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:53.805125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 629/1268] 10.222.0.45 () {56 vars in 1143 bytes} [Thu Aug 4 02:07:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:53.810893 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cece8-4e96-4d66-84a9-b8bfc49b707c None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:53.907148 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1cece8-4e96-4d66-84a9-b8bfc49b707c None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:53.940407 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6af4b-2fd3-42fa-97f5-9d7f8972c177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:53.941067 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6af4b-2fd3-42fa-97f5-9d7f8972c177 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:53.941346 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6af4b-2fd3-42fa-97f5-9d7f8972c177 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:53.941615 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6af4b-2fd3-42fa-97f5-9d7f8972c177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:53.951971 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6af4b-2fd3-42fa-97f5-9d7f8972c177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:53.953417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6af4b-2fd3-42fa-97f5-9d7f8972c177 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:53.967741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-f0e6af4b-2fd3-42fa-97f5-9d7f8972c177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:53.967741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:53.967741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:53.967741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:53.967741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:53.967741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:53.967741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:53.967741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:53.967741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:53.967741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:53.967741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:53.967741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:53.967741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:07:53.967741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:07:53.967741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:07:53.968854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:07:53.968854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:53.968854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:53.968854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:53.968854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:53.968854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:53.968854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:53.968854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:53.968854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:53.968854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:53.968854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:53.968854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:53.968854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:53.968854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:53.968854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:53.968854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:53.968854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:53.969974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:07:53.969974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:07:53.969974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:53.969974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:07:53.969974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 640/1269] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:53.975667 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8860703-3a84-4944-a91e-a8b1f42b1c22 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:54.071704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8860703-3a84-4944-a91e-a8b1f42b1c22 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:54.105313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46e281-4355-4cdf-a822-fe5bdafb042b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:54.106046 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46e281-4355-4cdf-a822-fe5bdafb042b None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:54.106324 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46e281-4355-4cdf-a822-fe5bdafb042b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:54.106593 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46e281-4355-4cdf-a822-fe5bdafb042b None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:54.107299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46e281-4355-4cdf-a822-fe5bdafb042b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:54.107622 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46e281-4355-4cdf-a822-fe5bdafb042b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:54.108990 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46e281-4355-4cdf-a822-fe5bdafb042b None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:54.123160 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 630/1270] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:07:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:54.129531 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6af4b-2fd3-42fa-97f5-9d7f8972c177 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:54.162987 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:54.231021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6af4b-2fd3-42fa-97f5-9d7f8972c177 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:54.263218 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7483f1-0e07-406e-a0f4-81a5aad7e61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:54.263843 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7483f1-0e07-406e-a0f4-81a5aad7e61a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:54.264121 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7483f1-0e07-406e-a0f4-81a5aad7e61a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:54.264392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7483f1-0e07-406e-a0f4-81a5aad7e61a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:54.271411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7483f1-0e07-406e-a0f4-81a5aad7e61a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:54.272862 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7483f1-0e07-406e-a0f4-81a5aad7e61a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:54.284400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-6d7483f1-0e07-406e-a0f4-81a5aad7e61a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:07:54.284400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:54.284400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:54.284400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:54.284400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:54.284400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:54.284400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:54.284400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:54.284400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:54.284400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:54.284400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:54.284400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:54.284400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:07:54.284400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:07:54.284400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:07:54.285504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:07:54.285504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:54.285504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:54.285504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:54.285504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:54.285504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:54.285504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:54.285504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:54.285504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:54.285504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:54.285504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:54.285504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:54.285504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:54.285504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:54.285504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:54.285504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:54.285504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:54.286622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:07:54.286622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:07:54.286622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:07:54.286622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:07:54.286622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:07:54.286622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:07:54.286622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:07:54.286622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:07:54.286622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 641/1271] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:07:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:54.291451 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46e281-4355-4cdf-a822-fe5bdafb042b None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:54.356805 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:54.397806 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46e281-4355-4cdf-a822-fe5bdafb042b None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:54.431260 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30c41a-aa5f-4b62-8600-bb37d58f905d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:54.431878 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30c41a-aa5f-4b62-8600-bb37d58f905d None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:54.432159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30c41a-aa5f-4b62-8600-bb37d58f905d None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:54.432435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30c41a-aa5f-4b62-8600-bb37d58f905d None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:54.433212 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30c41a-aa5f-4b62-8600-bb37d58f905d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:54.433521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30c41a-aa5f-4b62-8600-bb37d58f905d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:54.434956 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f30c41a-aa5f-4b62-8600-bb37d58f905d None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:54.447497 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 631/1272] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:07:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:54.453545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7483f1-0e07-406e-a0f4-81a5aad7e61a None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:54.545695 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:54.554635 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7483f1-0e07-406e-a0f4-81a5aad7e61a None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:54.587835 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985e0d8-c4f6-467c-a2d1-68d8d324609d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:54.588482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985e0d8-c4f6-467c-a2d1-68d8d324609d None admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:54.588788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985e0d8-c4f6-467c-a2d1-68d8d324609d None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:54.589059 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985e0d8-c4f6-467c-a2d1-68d8d324609d None admin] PATH_INFO: `/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/796772cf491047c8b0a609dae9402307/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:54.590946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985e0d8-c4f6-467c-a2d1-68d8d324609d None admin] RBAC: Authorizing `identity:create_grant(project_id=74b8caff828a4f5289094fb2ae2278f7, user_id=796772cf491047c8b0a609dae9402307, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:54.592354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985e0d8-c4f6-467c-a2d1-68d8d324609d None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:54.604742 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 642/1273] 10.222.0.45 () {58 vars in 1439 bytes} [Thu Aug 4 02:07:54 2022] PUT /identity/v3/projects/74b8caff828a4f5289094fb2ae2278f7/users/796772cf491047c8b0a609dae9402307/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:07:56.586566 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2cace-f820-42db-903f-33f79d73ebfe None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:56.587097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2cace-f820-42db-903f-33f79d73ebfe None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:56.587215 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e2cace-f820-42db-903f-33f79d73ebfe None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:56.588496 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 632/1274] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:07:56.593757 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405f7e5-186c-4a75-831e-a7330ed91bfb None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:56.594166 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405f7e5-186c-4a75-831e-a7330ed91bfb None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:56.594298 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1405f7e5-186c-4a75-831e-a7330ed91bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:56.612991 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-1405f7e5-186c-4a75-831e-a7330ed91bfb None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:07:56.614157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1405f7e5-186c-4a75-831e-a7330ed91bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:07:56.679887 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 643/1275] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:07:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:07:56.702785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2cace-f820-42db-903f-33f79d73ebfe None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:56.801790 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e2cace-f820-42db-903f-33f79d73ebfe None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:56.819163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:56.836392 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf112ee8-510c-4f0e-9725-f4bce5b351f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:56.837105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf112ee8-510c-4f0e-9725-f4bce5b351f5 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:56.837332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf112ee8-510c-4f0e-9725-f4bce5b351f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:56.837603 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf112ee8-510c-4f0e-9725-f4bce5b351f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:56.847927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf112ee8-510c-4f0e-9725-f4bce5b351f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 04 02:07:56.852757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-bf112ee8-510c-4f0e-9725-f4bce5b351f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:07:56.852757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:56.852757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:56.852757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:56.852757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:56.852757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:56.852757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:56.852757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:56.852757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:56.852757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:56.852757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:56.852757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:56.852757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 04 02:07:56.852757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 04 02:07:56.852757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 04 02:07:56.852757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 04 02:07:56.853848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 04 02:07:56.853848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 04 02:07:56.853848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 04 02:07:56.853848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 04 02:07:56.853848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:56.853848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:56.853848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:56.853848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:56.853848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:56.853848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:56.853848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:56.853848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:56.853848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:56.853848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:56.853848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:56.853848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:56.854701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:56.854701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:56.854701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:56.854701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:56.854701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 04 02:07:56.854701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 04 02:07:56.854701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 04 02:07:56.854701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 04 02:07:56.854701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 04 02:07:56.854701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 04 02:07:56.854701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 04 02:07:56.854701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:07:56.854701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 633/1276] 10.222.0.45 () {56 vars in 1096 bytes} [Thu Aug 4 02:07:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:56.860216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405f7e5-186c-4a75-831e-a7330ed91bfb None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:56.938662 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:56.958389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1405f7e5-186c-4a75-831e-a7330ed91bfb None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:56.991577 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e12231-17d0-486d-8222-69727c5dd569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:56.992191 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e12231-17d0-486d-8222-69727c5dd569 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:56.992474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e12231-17d0-486d-8222-69727c5dd569 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:56.992778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e12231-17d0-486d-8222-69727c5dd569 None admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:56.993491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e12231-17d0-486d-8222-69727c5dd569 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:56.993795 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e12231-17d0-486d-8222-69727c5dd569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:56.995158 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e12231-17d0-486d-8222-69727c5dd569 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:57.003107 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 644/1277] 10.222.0.45 () {56 vars in 1099 bytes} [Thu Aug 4 02:07:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:57.008991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf112ee8-510c-4f0e-9725-f4bce5b351f5 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:57.110819 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf112ee8-510c-4f0e-9725-f4bce5b351f5 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:57.143108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7957a-3ddc-4620-b53f-0e45baa76e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:57.143726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7957a-3ddc-4620-b53f-0e45baa76e4d None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:57.143997 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7957a-3ddc-4620-b53f-0e45baa76e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:57.144262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7957a-3ddc-4620-b53f-0e45baa76e4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:57.153729 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7957a-3ddc-4620-b53f-0e45baa76e4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:57.155173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7957a-3ddc-4620-b53f-0e45baa76e4d None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:57.169624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-84b7957a-3ddc-4620-b53f-0e45baa76e4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:57.169624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:57.169624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:57.169624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:57.169624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:57.169624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:57.169624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:57.169624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:57.169624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:57.169624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:57.169624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:57.169624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:57.169624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:07:57.169624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:07:57.169624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:07:57.170730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:07:57.170730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:57.170730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:57.170730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:57.170730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:57.170730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:57.170730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:57.170730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:57.170730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:57.170730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:57.170730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:57.170730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:57.170730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:57.170730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:57.170730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:57.170730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:57.170730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:57.171741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:07:57.171741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:07:57.171741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:57.171741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:07:57.171741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 634/1278] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:57.177169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e12231-17d0-486d-8222-69727c5dd569 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:57.273624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e12231-17d0-486d-8222-69727c5dd569 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:57.308330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590ac22-a89a-4f2f-ac1d-70b428430635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:57.309070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590ac22-a89a-4f2f-ac1d-70b428430635 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:57.309368 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590ac22-a89a-4f2f-ac1d-70b428430635 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:57.309636 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590ac22-a89a-4f2f-ac1d-70b428430635 None admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:57.310574 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590ac22-a89a-4f2f-ac1d-70b428430635 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:57.310897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590ac22-a89a-4f2f-ac1d-70b428430635 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:57.312490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8590ac22-a89a-4f2f-ac1d-70b428430635 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:57.331439 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 645/1279] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:57.332680 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7957a-3ddc-4620-b53f-0e45baa76e4d None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:57.368009 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:57.436137 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7957a-3ddc-4620-b53f-0e45baa76e4d None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:57.467769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde3606-4766-4966-9a84-ed82c7ae55f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:57.468381 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde3606-4766-4966-9a84-ed82c7ae55f1 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:57.468676 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde3606-4766-4966-9a84-ed82c7ae55f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:57.468943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde3606-4766-4966-9a84-ed82c7ae55f1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:57.476482 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde3606-4766-4966-9a84-ed82c7ae55f1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:57.477927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde3606-4766-4966-9a84-ed82c7ae55f1 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:57.489299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-dbde3606-4766-4966-9a84-ed82c7ae55f1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 04 02:07:57.489299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:57.489299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:57.489299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:57.489299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:57.489299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:57.489299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:57.489299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:57.489299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:57.489299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:57.489299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:57.489299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:57.489299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 04 02:07:57.489299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 04 02:07:57.489299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 04 02:07:57.489299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:57.490436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:57.491800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:57.491800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:57.491800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 04 02:07:57.491800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 04 02:07:57.491800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 04 02:07:57.491800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 04 02:07:57.491800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 04 02:07:57.491800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 04 02:07:57.491800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 04 02:07:57.491800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:07:57.491800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 635/1280] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:57.496198 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590ac22-a89a-4f2f-ac1d-70b428430635 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:57.499969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:57.593869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590ac22-a89a-4f2f-ac1d-70b428430635 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:57.626780 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5228e-d3b6-4388-a593-c60fd4d2a56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:57.627410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5228e-d3b6-4388-a593-c60fd4d2a56a None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:57.627691 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5228e-d3b6-4388-a593-c60fd4d2a56a None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:57.627964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5228e-d3b6-4388-a593-c60fd4d2a56a None admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:57.628795 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec5228e-d3b6-4388-a593-c60fd4d2a56a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:57.629104 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec5228e-d3b6-4388-a593-c60fd4d2a56a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:57.630534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec5228e-d3b6-4388-a593-c60fd4d2a56a None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:57.648355 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 646/1281] 10.222.0.45 () {56 vars in 1143 bytes} [Thu Aug 4 02:07:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:57.654041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde3606-4766-4966-9a84-ed82c7ae55f1 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:57.741741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:57.751450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde3606-4766-4966-9a84-ed82c7ae55f1 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:57.785480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad090e-430f-42ce-a773-236f39a61c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:57.786082 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad090e-430f-42ce-a773-236f39a61c0d None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:57.786356 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad090e-430f-42ce-a773-236f39a61c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:57.786620 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad090e-430f-42ce-a773-236f39a61c0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:57.850396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad090e-430f-42ce-a773-236f39a61c0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:57.851826 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad090e-430f-42ce-a773-236f39a61c0d None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:57.936359 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:57.946475 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 636/1282] 10.222.0.45 () {58 vars in 1282 bytes} [Thu Aug 4 02:07:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 04 02:07:57.952254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5228e-d3b6-4388-a593-c60fd4d2a56a None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:58.012317 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:58.050140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5228e-d3b6-4388-a593-c60fd4d2a56a None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:58.084568 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cd8f8-31e5-436c-93c8-f17241d57a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:58.085245 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cd8f8-31e5-436c-93c8-f17241d57a5b None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:58.085527 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cd8f8-31e5-436c-93c8-f17241d57a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:58.085799 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84cd8f8-31e5-436c-93c8-f17241d57a5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:58.095933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84cd8f8-31e5-436c-93c8-f17241d57a5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:58.097419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84cd8f8-31e5-436c-93c8-f17241d57a5b None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:58.111779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-e84cd8f8-31e5-436c-93c8-f17241d57a5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:58.111779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:58.111779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:58.111779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:58.111779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:58.111779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:58.111779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:58.111779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:58.111779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:58.111779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:58.111779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:58.111779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:58.111779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 04 02:07:58.111779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 04 02:07:58.111779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 04 02:07:58.112859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 04 02:07:58.112859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:58.112859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:58.112859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:58.112859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:58.112859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:58.112859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:58.112859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:58.112859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:58.112859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:58.112859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:58.112859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:58.112859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:58.112859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:58.112859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:58.112859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:58.112859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:58.113993 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 04 02:07:58.113993 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 04 02:07:58.113993 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 04 02:07:58.113993 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:07:58.113993 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 647/1283] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:58.118678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad090e-430f-42ce-a773-236f39a61c0d None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:58.215183 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad090e-430f-42ce-a773-236f39a61c0d None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:58.250049 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e4c07-e1d4-4cf3-90aa-83c86653399b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:58.250654 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e4c07-e1d4-4cf3-90aa-83c86653399b None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:58.250931 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e4c07-e1d4-4cf3-90aa-83c86653399b None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:58.251198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e4c07-e1d4-4cf3-90aa-83c86653399b None admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:58.251874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e4c07-e1d4-4cf3-90aa-83c86653399b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:58.252704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e4c07-e1d4-4cf3-90aa-83c86653399b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:58.254072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e4c07-e1d4-4cf3-90aa-83c86653399b None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:58.264824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 637/1284] 10.222.0.45 () {56 vars in 1109 bytes} [Thu Aug 4 02:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:58.271129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cd8f8-31e5-436c-93c8-f17241d57a5b None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:58.372637 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84cd8f8-31e5-436c-93c8-f17241d57a5b None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:58.386275 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:58.406060 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20aea9e-91a2-4d65-8ad4-604241851477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:58.406689 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20aea9e-91a2-4d65-8ad4-604241851477 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:58.406970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20aea9e-91a2-4d65-8ad4-604241851477 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:58.407245 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20aea9e-91a2-4d65-8ad4-604241851477 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:58.413788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20aea9e-91a2-4d65-8ad4-604241851477 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:58.415213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20aea9e-91a2-4d65-8ad4-604241851477 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:58.426777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application [None req-f20aea9e-91a2-4d65-8ad4-604241851477 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:07:58.426777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:07:58.426777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:07:58.426777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:07:58.426777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:07:58.426777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:07:58.426777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:07:58.426777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:07:58.426777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:07:58.426777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:07:58.426777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:07:58.426777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:07:58.426777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 04 02:07:58.426777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 04 02:07:58.426777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 04 02:07:58.427831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 04 02:07:58.427831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:07:58.427831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:07:58.427831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:07:58.427831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:07:58.427831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:07:58.427831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:07:58.427831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:07:58.427831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application with Lock( Aug 04 02:07:58.427831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:07:58.427831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:07:58.427831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:07:58.427831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:07:58.427831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:07:58.427831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:07:58.427831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:07:58.427831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:07:58.429008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 04 02:07:58.429008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 04 02:07:58.429008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 04 02:07:58.429008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 04 02:07:58.429008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 04 02:07:58.429008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 04 02:07:58.429008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 04 02:07:58.429008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: ERROR keystone.server.flask.application Aug 04 02:07:58.429008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 648/1285] 10.222.0.45 () {56 vars in 1111 bytes} [Thu Aug 4 02:07:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:07:58.433767 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e4c07-e1d4-4cf3-90aa-83c86653399b None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:58.481648 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:07:58.531706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e4c07-e1d4-4cf3-90aa-83c86653399b None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:58.563294 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f06848-a045-43c0-9a84-b21d67672aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:58.563889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f06848-a045-43c0-9a84-b21d67672aba None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:58.564164 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f06848-a045-43c0-9a84-b21d67672aba None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:58.565099 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f06848-a045-43c0-9a84-b21d67672aba None admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:58.565903 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f06848-a045-43c0-9a84-b21d67672aba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:58.566202 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f06848-a045-43c0-9a84-b21d67672aba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:58.567615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f06848-a045-43c0-9a84-b21d67672aba None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:58.579290 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 638/1286] 10.222.0.45 () {56 vars in 1148 bytes} [Thu Aug 4 02:07:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:07:58.585401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20aea9e-91a2-4d65-8ad4-604241851477 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:07:58.683602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20aea9e-91a2-4d65-8ad4-604241851477 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:07:58.716698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5217a-69d7-4b1b-859b-5e4418b83ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:07:58.717297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5217a-69d7-4b1b-859b-5e4418b83ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:07:58.717579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5217a-69d7-4b1b-859b-5e4418b83ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:07:58.717934 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5217a-69d7-4b1b-859b-5e4418b83ff1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:07:58.718629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5217a-69d7-4b1b-859b-5e4418b83ff1 None admin] RBAC: Adding query filter params (role.id=35205394c8af4c52a61b12f92ba16644, scope.project.id=74b8caff828a4f5289094fb2ae2278f7, user.id=796772cf491047c8b0a609dae9402307) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:07:58.718939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5217a-69d7-4b1b-859b-5e4418b83ff1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:07:58.720351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5217a-69d7-4b1b-859b-5e4418b83ff1 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:07:58.726627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 649/1287] 10.222.0.45 () {56 vars in 1374 bytes} [Thu Aug 4 02:07:58 2022] GET /identity/v3/role_assignments?user.id=796772cf491047c8b0a609dae9402307&scope.project.id=74b8caff828a4f5289094fb2ae2278f7&role.id=35205394c8af4c52a61b12f92ba16644 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:08:00.739505 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fec9f-cdb5-49d2-9fd1-73455d699fec None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:00.740040 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fec9f-cdb5-49d2-9fd1-73455d699fec None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:00.740040 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099fec9f-cdb5-49d2-9fd1-73455d699fec None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:00.741436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 639/1288] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:08:00.746900 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305e9b4-063a-46dc-bd69-d9750a8ed3b2 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:00.747310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305e9b4-063a-46dc-bd69-d9750a8ed3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:00.747456 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305e9b4-063a-46dc-bd69-d9750a8ed3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:00.765139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-3305e9b4-063a-46dc-bd69-d9750a8ed3b2 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:08:00.766325 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-3305e9b4-063a-46dc-bd69-d9750a8ed3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:08:00.796712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:08:00.833943 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 650/1289] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:08:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:08:00.856457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fec9f-cdb5-49d2-9fd1-73455d699fec None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:08:00.954442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099fec9f-cdb5-49d2-9fd1-73455d699fec None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:08:00.965090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:08:00.988399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e704e1-cfa9-4c94-a18c-5c5f7326ec1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:08:00.989061 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e704e1-cfa9-4c94-a18c-5c5f7326ec1c None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:00.989345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e704e1-cfa9-4c94-a18c-5c5f7326ec1c None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:00.989613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e704e1-cfa9-4c94-a18c-5c5f7326ec1c None admin] PATH_INFO: `/v3/services/placement` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:00.990239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e704e1-cfa9-4c94-a18c-5c5f7326ec1c None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:08:00.991598 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e704e1-cfa9-4c94-a18c-5c5f7326ec1c None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:08:01.000242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-c9e704e1-cfa9-4c94-a18c-5c5f7326ec1c None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 02:08:01.000242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:08:01.000242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:08:01.000242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:08:01.000242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:08:01.000242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:08:01.000242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:08:01.000242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:08:01.000242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 02:08:01.000242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:08:01.000242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:08:01.000242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:08:01.000242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:08:01.000242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:08:01.000242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:08:01.000242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:08:01.000242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:08:01.001450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:08:01.001450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:08:01.001450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:08:01.001450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:08:01.001450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:08:01.001450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:08:01.001450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:08:01.001450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:08:01.001450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:08:01.001450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:08:01.001450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:08:01.001450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:08:01.001450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:08:01.001450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:08:01.001450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:08:01.001450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:08:01.002812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:08:01.002812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:08:01.002812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:08:01.002812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:08:01.002812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:08:01.002812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:08:01.002812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:08:01.002812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:08:01.002812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:08:01.002812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:08:01.002812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:08:01.002812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:08:01.002812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 02:08:01.002812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:08:01.002812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 640/1290] 10.222.0.45 () {56 vars in 1117 bytes} [Thu Aug 4 02:08:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:08:01.008157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305e9b4-063a-46dc-bd69-d9750a8ed3b2 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:08:01.105792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305e9b4-063a-46dc-bd69-d9750a8ed3b2 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:08:01.140383 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20505be6-cb2f-4c89-9a6a-aa654cf08263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:08:01.141016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20505be6-cb2f-4c89-9a6a-aa654cf08263 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:01.141299 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20505be6-cb2f-4c89-9a6a-aa654cf08263 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:01.141569 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20505be6-cb2f-4c89-9a6a-aa654cf08263 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:01.142242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20505be6-cb2f-4c89-9a6a-aa654cf08263 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:08:01.143103 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20505be6-cb2f-4c89-9a6a-aa654cf08263 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:08:01.144465 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20505be6-cb2f-4c89-9a6a-aa654cf08263 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:08:01.149227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 651/1291] 10.222.0.45 () {56 vars in 1116 bytes} [Thu Aug 4 02:08:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:08:01.155268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e704e1-cfa9-4c94-a18c-5c5f7326ec1c None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:08:01.250140 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e704e1-cfa9-4c94-a18c-5c5f7326ec1c None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:08:01.282430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac4b9e-b3f0-4ddb-9232-c7860b47810f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:08:01.283045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac4b9e-b3f0-4ddb-9232-c7860b47810f None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:01.283326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac4b9e-b3f0-4ddb-9232-c7860b47810f None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:01.283600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac4b9e-b3f0-4ddb-9232-c7860b47810f None admin] PATH_INFO: `/v3/services` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:01.284274 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac4b9e-b3f0-4ddb-9232-c7860b47810f None admin] RBAC: Adding query filter params (type=placement) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:08:01.284578 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac4b9e-b3f0-4ddb-9232-c7860b47810f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:08:01.285973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac4b9e-b3f0-4ddb-9232-c7860b47810f None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:08:01.290295 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 641/1292] 10.222.0.45 () {56 vars in 1116 bytes} [Thu Aug 4 02:08:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:08:03.304911 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd68bb-7394-4312-a0e7-e42bd82a1349 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:03.305591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd68bb-7394-4312-a0e7-e42bd82a1349 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:03.305591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd68bb-7394-4312-a0e7-e42bd82a1349 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:03.306853 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 652/1293] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:08:03.312058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e10a2a-f173-4f78-bf42-87ac3a9d93ec None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:03.312785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e10a2a-f173-4f78-bf42-87ac3a9d93ec None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:03.312785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e10a2a-f173-4f78-bf42-87ac3a9d93ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:03.333915 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-53e10a2a-f173-4f78-bf42-87ac3a9d93ec None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:08:03.335164 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-53e10a2a-f173-4f78-bf42-87ac3a9d93ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:08:03.375411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:08:03.417285 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 642/1294] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:08:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:08:03.439763 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd68bb-7394-4312-a0e7-e42bd82a1349 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:08:03.471718 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:08:03.541069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd68bb-7394-4312-a0e7-e42bd82a1349 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:08:03.582704 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5ebd4-b8e7-4c32-9931-79e66f1736b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:08:03.583229 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5ebd4-b8e7-4c32-9931-79e66f1736b9 None admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:03.583507 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5ebd4-b8e7-4c32-9931-79e66f1736b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:03.583779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5ebd4-b8e7-4c32-9931-79e66f1736b9 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:03.584687 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5ebd4-b8e7-4c32-9931-79e66f1736b9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:08:03.586051 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5ebd4-b8e7-4c32-9931-79e66f1736b9 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:08:03.603718 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 653/1295] 10.222.0.45 () {60 vars in 1141 bytes} [Thu Aug 4 02:08:03 2022] POST /identity/v3/services => generated 246 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:08:05.617207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719cf5f-12bd-41e7-8b70-423d0c1db722 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:05.617780 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719cf5f-12bd-41e7-8b70-423d0c1db722 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:05.617780 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719cf5f-12bd-41e7-8b70-423d0c1db722 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:05.619098 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 643/1296] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:08:05.624218 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbc439-ebe3-48aa-a74f-30ab9aa8b066 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:05.624656 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbc439-ebe3-48aa-a74f-30ab9aa8b066 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:05.624807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bbc439-ebe3-48aa-a74f-30ab9aa8b066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:05.643430 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-73bbc439-ebe3-48aa-a74f-30ab9aa8b066 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:08:05.644663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-73bbc439-ebe3-48aa-a74f-30ab9aa8b066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:08:05.715657 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 654/1297] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:08:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:08:05.737736 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719cf5f-12bd-41e7-8b70-423d0c1db722 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:08:05.836996 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719cf5f-12bd-41e7-8b70-423d0c1db722 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:08:05.869452 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be0927-8a82-43f0-91b3-1d9bc55450a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:08:05.870078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be0927-8a82-43f0-91b3-1d9bc55450a2 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:05.870356 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be0927-8a82-43f0-91b3-1d9bc55450a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:05.870627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be0927-8a82-43f0-91b3-1d9bc55450a2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:05.871244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be0927-8a82-43f0-91b3-1d9bc55450a2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:08:05.872599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be0927-8a82-43f0-91b3-1d9bc55450a2 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:08:05.881787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-c3be0927-8a82-43f0-91b3-1d9bc55450a2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 02:08:05.881787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:08:05.881787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:08:05.881787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:08:05.881787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:08:05.881787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:08:05.881787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:08:05.881787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:08:05.881787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 02:08:05.881787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:08:05.881787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:08:05.881787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:08:05.881787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:08:05.881787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:08:05.881787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:08:05.881787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:08:05.881787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:08:05.883053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:08:05.883053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:08:05.883053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:08:05.883053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:08:05.883053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:08:05.883053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:08:05.883053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:08:05.883053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:08:05.883053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:08:05.883053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:08:05.883053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:08:05.883053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:08:05.883053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:08:05.883053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:08:05.883053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:08:05.883053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:08:05.884190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:08:05.884190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:08:05.884190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:08:05.884190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:08:05.884190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:08:05.884190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:08:05.884190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:08:05.884190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:08:05.884190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:08:05.884190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:08:05.884190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:08:05.884190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:08:05.884190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 02:08:05.884190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:08:05.884190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 644/1298] 10.222.0.45 () {56 vars in 1117 bytes} [Thu Aug 4 02:08:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:08:05.888932 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bbc439-ebe3-48aa-a74f-30ab9aa8b066 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:08:05.907321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:08:05.996510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bbc439-ebe3-48aa-a74f-30ab9aa8b066 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:08:06.029366 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc174041-7b05-4a1f-8d98-5be59118f355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:08:06.029965 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc174041-7b05-4a1f-8d98-5be59118f355 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:06.030239 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc174041-7b05-4a1f-8d98-5be59118f355 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:06.030509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc174041-7b05-4a1f-8d98-5be59118f355 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:06.031173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc174041-7b05-4a1f-8d98-5be59118f355 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:08:06.031473 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc174041-7b05-4a1f-8d98-5be59118f355 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:08:06.032840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc174041-7b05-4a1f-8d98-5be59118f355 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:08:06.038438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 655/1299] 10.222.0.45 () {56 vars in 1116 bytes} [Thu Aug 4 02:08:05 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:08:06.044383 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be0927-8a82-43f0-91b3-1d9bc55450a2 None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:08:06.076874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:08:06.142903 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be0927-8a82-43f0-91b3-1d9bc55450a2 None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:08:06.177154 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1afba-19c3-42ab-9df9-d1e77c270288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:08:06.177787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1afba-19c3-42ab-9df9-d1e77c270288 None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:06.178066 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1afba-19c3-42ab-9df9-d1e77c270288 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:06.178337 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1afba-19c3-42ab-9df9-d1e77c270288 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:06.179091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1afba-19c3-42ab-9df9-d1e77c270288 None admin] RBAC: Adding query filter params (interface=public, service_id=d637bcc2b41248429b720aa67a490b47, region_id=RegionOne) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:08:06.179400 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1afba-19c3-42ab-9df9-d1e77c270288 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:08:06.180786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1afba-19c3-42ab-9df9-d1e77c270288 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:08:06.185702 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 645/1300] 10.222.0.45 () {56 vars in 1251 bytes} [Thu Aug 4 02:08:06 2022] GET /identity/v3/endpoints?service_id=d637bcc2b41248429b720aa67a490b47&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:08:06.192016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc174041-7b05-4a1f-8d98-5be59118f355 None admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:08:06.272273 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:08:06.296160 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc174041-7b05-4a1f-8d98-5be59118f355 None admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:08:06.337307 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c5b45a-52d3-42cc-9347-53c628faadcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:08:06.338066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5b45a-52d3-42cc-9347-53c628faadcc None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:06.338414 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5b45a-52d3-42cc-9347-53c628faadcc None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:06.338890 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c5b45a-52d3-42cc-9347-53c628faadcc None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:06.339500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5b45a-52d3-42cc-9347-53c628faadcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:08:06.341298 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c5b45a-52d3-42cc-9347-53c628faadcc None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:08:06.349803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 656/1301] 10.222.0.45 () {56 vars in 1087 bytes} [Thu Aug 4 02:08:06 2022] GET /identity/v3/services => generated 2191 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:08:08.352509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a9356-3671-4359-b36f-9156273dea80 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:08.353112 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a9356-3671-4359-b36f-9156273dea80 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:08.353112 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a9356-3671-4359-b36f-9156273dea80 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:08.354432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 646/1302] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:08:08.359426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01e92c-a4a7-4790-9b16-acfee6b3db4b None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:08.359809 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01e92c-a4a7-4790-9b16-acfee6b3db4b None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:08.359975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01e92c-a4a7-4790-9b16-acfee6b3db4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:08.378747 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-9f01e92c-a4a7-4790-9b16-acfee6b3db4b None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:08:08.379935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9f01e92c-a4a7-4790-9b16-acfee6b3db4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:08:08.444206 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 657/1303] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:08:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 04 02:08:08.466752 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a9356-3671-4359-b36f-9156273dea80 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:08:08.563510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:08:08.570747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a9356-3671-4359-b36f-9156273dea80 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:08:08.602937 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b58a9d-ddc8-4260-8022-ef8d3b8e663d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:08:08.603509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b58a9d-ddc8-4260-8022-ef8d3b8e663d None admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:08.603789 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b58a9d-ddc8-4260-8022-ef8d3b8e663d None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:08.604058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b58a9d-ddc8-4260-8022-ef8d3b8e663d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:08.604704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b58a9d-ddc8-4260-8022-ef8d3b8e663d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:08:08.606072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b58a9d-ddc8-4260-8022-ef8d3b8e663d None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:08:08.615370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application [None req-99b58a9d-ddc8-4260-8022-ef8d3b8e663d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 02:08:08.615370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:08:08.615370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 04 02:08:08.615370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 04 02:08:08.615370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 04 02:08:08.615370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 04 02:08:08.615370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 04 02:08:08.615370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:08:08.615370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 02:08:08.615370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:08:08.615370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 04 02:08:08.615370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:08:08.615370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 04 02:08:08.615370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 04 02:08:08.615370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 04 02:08:08.615370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 04 02:08:08.615370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 04 02:08:08.616550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 04 02:08:08.616550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 04 02:08:08.616550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 04 02:08:08.616550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 04 02:08:08.616550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 04 02:08:08.616550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 04 02:08:08.616550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 04 02:08:08.616550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 04 02:08:08.616550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 04 02:08:08.616550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 04 02:08:08.616550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 04 02:08:08.616550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 04 02:08:08.616550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 04 02:08:08.616550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 04 02:08:08.616550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 04 02:08:08.616550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.get_or_create( Aug 04 02:08:08.617606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 04 02:08:08.617606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application with Lock( Aug 04 02:08:08.617606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 04 02:08:08.617606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self._enter() Aug 04 02:08:08.617606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 04 02:08:08.617606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 04 02:08:08.617606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 04 02:08:08.617606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application return self.creator() Aug 04 02:08:08.617606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 04 02:08:08.617606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application created_value = creator( Aug 04 02:08:08.617606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 04 02:08:08.617606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 04 02:08:08.617606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 04 02:08:08.617606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: ERROR keystone.server.flask.application Aug 04 02:08:08.617606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 647/1304] 10.222.0.45 () {56 vars in 1117 bytes} [Thu Aug 4 02:08:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 04 02:08:08.622595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01e92c-a4a7-4790-9b16-acfee6b3db4b None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:08:08.697435 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:08:08.719789 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01e92c-a4a7-4790-9b16-acfee6b3db4b None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:08:08.754101 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ffb02-8c44-406b-9e7a-999a86dada41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:08:08.754708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ffb02-8c44-406b-9e7a-999a86dada41 None admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:08.754990 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ffb02-8c44-406b-9e7a-999a86dada41 None admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:08.755263 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ffb02-8c44-406b-9e7a-999a86dada41 None admin] PATH_INFO: `/v3/services` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:08.755937 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ffb02-8c44-406b-9e7a-999a86dada41 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:08:08.756236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ffb02-8c44-406b-9e7a-999a86dada41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:08:08.757624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ffb02-8c44-406b-9e7a-999a86dada41 None admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:08:08.762629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 658/1305] 10.222.0.45 () {56 vars in 1116 bytes} [Thu Aug 4 02:08:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:08:08.768437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b58a9d-ddc8-4260-8022-ef8d3b8e663d None admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:08:08.864443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b58a9d-ddc8-4260-8022-ef8d3b8e663d None admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:08:08.904305 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c0685-1ff1-4864-9c04-102d0f33dff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:08:08.904961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0685-1ff1-4864-9c04-102d0f33dff7 None admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:08:08.905241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0685-1ff1-4864-9c04-102d0f33dff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:08:08.905513 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0685-1ff1-4864-9c04-102d0f33dff7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:08:08.906348 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c0685-1ff1-4864-9c04-102d0f33dff7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:08:08.907724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c0685-1ff1-4864-9c04-102d0f33dff7 None admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:08:08.927617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 648/1306] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:08:08 2022] POST /identity/v3/endpoints => generated 331 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:08:18.760097 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=30 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:08:18.923932 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:12.382108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3182d96-470c-48b5-ba27-eaa9235b1932 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:12.382700 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3182d96-470c-48b5-ba27-eaa9235b1932 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:12.382700 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3182d96-470c-48b5-ba27-eaa9235b1932 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:12.389345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dd79f-d63c-4193-9f1b-33acbb7871b7 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:12.389842 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dd79f-d63c-4193-9f1b-33acbb7871b7 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:12.389938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5dd79f-d63c-4193-9f1b-33acbb7871b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:12.391822 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 659/1307] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:09:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:09:12.509746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-fa5dd79f-d63c-4193-9f1b-33acbb7871b7 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:09:12.512900 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-fa5dd79f-d63c-4193-9f1b-33acbb7871b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:12.694572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 649/1308] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:09:12.745165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e3493-e4f7-4709-b2ee-f20070520cc2 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:12.745455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e3493-e4f7-4709-b2ee-f20070520cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:12.745735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e3493-e4f7-4709-b2ee-f20070520cc2 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:12.747232 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 660/1309] 10.222.0.45 () {54 vars in 965 bytes} [Thu Aug 4 02:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:09:12.753266 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd30d7-b6fa-440f-96f3-96c18290a3e1 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:12.753761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd30d7-b6fa-440f-96f3-96c18290a3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:12.754041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd30d7-b6fa-440f-96f3-96c18290a3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:12.836854 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-21bd30d7-b6fa-440f-96f3-96c18290a3e1 None None] MFA Rules not processed for user `6c2da427acb24c8cb6d667c6347858c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:09:12.837984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-21bd30d7-b6fa-440f-96f3-96c18290a3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:12.861155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:12.921671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 650/1310] 10.222.0.45 () {58 vars in 1062 bytes} [Thu Aug 4 02:09:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:09:12.930654 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e3493-e4f7-4709-b2ee-f20070520cc2 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:09:12.932537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-6b4e3493-e4f7-4709-b2ee-f20070520cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:13.055752 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e3493-e4f7-4709-b2ee-f20070520cc2 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:09:13.142525 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f574ad2-db9a-4a53-ac21-7ef4d73ac342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:09:13.143161 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f574ad2-db9a-4a53-ac21-7ef4d73ac342 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:13.143999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f574ad2-db9a-4a53-ac21-7ef4d73ac342 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:13.144271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f574ad2-db9a-4a53-ac21-7ef4d73ac342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:13.146056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4f574ad2-db9a-4a53-ac21-7ef4d73ac342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:13.208427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:13.272327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f574ad2-db9a-4a53-ac21-7ef4d73ac342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:09:13.273878 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f574ad2-db9a-4a53-ac21-7ef4d73ac342 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:09:13.294200 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 661/1311] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:09:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:09:15.665710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b567750-ff97-434b-b429-16839f21e384 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:15.666227 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b567750-ff97-434b-b429-16839f21e384 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:15.666320 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b567750-ff97-434b-b429-16839f21e384 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:15.667756 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 651/1312] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:09:15.673041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477458b4-2658-4569-8d84-80db8820283f None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:15.673478 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477458b4-2658-4569-8d84-80db8820283f None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:15.673735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477458b4-2658-4569-8d84-80db8820283f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:15.767951 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-477458b4-2658-4569-8d84-80db8820283f None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:09:15.769660 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-477458b4-2658-4569-8d84-80db8820283f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:15.886208 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 662/1313] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:09:15.924908 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b567750-ff97-434b-b429-16839f21e384 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:09:15.926443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-4b567750-ff97-434b-b429-16839f21e384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:16.035719 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b567750-ff97-434b-b429-16839f21e384 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:09:16.109745 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:16.113496 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bef03d-488a-4073-a8a9-f3bc1f937984 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:09:16.114353 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bef03d-488a-4073-a8a9-f3bc1f937984 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:16.114972 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bef03d-488a-4073-a8a9-f3bc1f937984 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:16.115311 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bef03d-488a-4073-a8a9-f3bc1f937984 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:16.117103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d4bef03d-488a-4073-a8a9-f3bc1f937984 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:16.232131 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bef03d-488a-4073-a8a9-f3bc1f937984 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:09:16.233615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bef03d-488a-4073-a8a9-f3bc1f937984 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:09:16.251763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 652/1314] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:09:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:09:18.623214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbf8c7-0bf5-4ea6-9cfd-49b9e2091a4c None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:18.623765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbf8c7-0bf5-4ea6-9cfd-49b9e2091a4c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:18.623765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbf8c7-0bf5-4ea6-9cfd-49b9e2091a4c None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:18.625157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 663/1315] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:09:18.630546 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9436c-765c-4203-9148-b4696187e668 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:18.630992 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9436c-765c-4203-9148-b4696187e668 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:18.631085 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c9436c-765c-4203-9148-b4696187e668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:18.698694 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-e9c9436c-765c-4203-9148-b4696187e668 None None] MFA Rules not processed for user `e88786550f274e61877748f90084c3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:09:18.699929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e9c9436c-765c-4203-9148-b4696187e668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:18.751835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:18.797048 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 653/1316] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:09:18 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:09:18.837613 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbf8c7-0bf5-4ea6-9cfd-49b9e2091a4c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:09:18.839278 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-bfcbf8c7-0bf5-4ea6-9cfd-49b9e2091a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:18.849712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:18.951189 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbf8c7-0bf5-4ea6-9cfd-49b9e2091a4c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:09:19.025588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253569d3-0076-4f7c-b77e-0ef9ae355bd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:09:19.026180 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253569d3-0076-4f7c-b77e-0ef9ae355bd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:19.026465 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253569d3-0076-4f7c-b77e-0ef9ae355bd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:19.026739 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253569d3-0076-4f7c-b77e-0ef9ae355bd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:19.028410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-253569d3-0076-4f7c-b77e-0ef9ae355bd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:19.091147 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:19.137719 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253569d3-0076-4f7c-b77e-0ef9ae355bd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:09:19.139139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253569d3-0076-4f7c-b77e-0ef9ae355bd1 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:09:19.157854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 664/1317] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:09:18 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:09:21.785449 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338c604-cbae-4ac1-8c69-d1329980c5a3 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:21.786008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338c604-cbae-4ac1-8c69-d1329980c5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:21.786008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338c604-cbae-4ac1-8c69-d1329980c5a3 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:21.787416 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 654/1318] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:09:21.792533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914710e3-aee0-49cd-9c50-fa9c6708902f None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:21.792960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914710e3-aee0-49cd-9c50-fa9c6708902f None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:21.793105 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914710e3-aee0-49cd-9c50-fa9c6708902f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:21.859666 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-914710e3-aee0-49cd-9c50-fa9c6708902f None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:09:21.860925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-914710e3-aee0-49cd-9c50-fa9c6708902f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:21.963200 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 665/1319] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:09:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:09:22.003925 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338c604-cbae-4ac1-8c69-d1329980c5a3 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:09:22.005503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d338c604-cbae-4ac1-8c69-d1329980c5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:22.120700 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338c604-cbae-4ac1-8c69-d1329980c5a3 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:09:22.196135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e94cb5-1168-4c96-8dac-23917f415c0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:09:22.197244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e94cb5-1168-4c96-8dac-23917f415c0a service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:22.199449 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e94cb5-1168-4c96-8dac-23917f415c0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:22.199449 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e94cb5-1168-4c96-8dac-23917f415c0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:22.200241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-09e94cb5-1168-4c96-8dac-23917f415c0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:22.208724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:22.312282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e94cb5-1168-4c96-8dac-23917f415c0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:09:22.314061 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e94cb5-1168-4c96-8dac-23917f415c0a service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:09:22.336379 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 655/1320] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:09:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:09:25.127011 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c325a-00a9-41b9-bd45-bbbd660af583 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:25.127571 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c325a-00a9-41b9-bd45-bbbd660af583 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:25.127908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c325a-00a9-41b9-bd45-bbbd660af583 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:25.129746 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 666/1321] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:09:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:09:25.135523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c62943-ea9a-43d8-aa44-9593fd582898 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:25.136112 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c62943-ea9a-43d8-aa44-9593fd582898 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:25.136112 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c62943-ea9a-43d8-aa44-9593fd582898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:25.208722 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-a1c62943-ea9a-43d8-aa44-9593fd582898 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:09:25.209958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a1c62943-ea9a-43d8-aa44-9593fd582898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:25.277232 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:25.326317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 656/1322] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:09:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:09:25.379572 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3f949-3213-4332-9934-7dc4a6a89eb7 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:25.379769 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3f949-3213-4332-9934-7dc4a6a89eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:25.380032 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3f949-3213-4332-9934-7dc4a6a89eb7 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:25.381514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 667/1323] 10.222.0.45 () {54 vars in 965 bytes} [Thu Aug 4 02:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:09:25.388434 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78282165-43e0-46b9-a3ff-b8c98d4443d3 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:25.388836 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78282165-43e0-46b9-a3ff-b8c98d4443d3 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:25.389180 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78282165-43e0-46b9-a3ff-b8c98d4443d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:25.477179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-78282165-43e0-46b9-a3ff-b8c98d4443d3 None None] MFA Rules not processed for user `6c2da427acb24c8cb6d667c6347858c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:09:25.478412 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-78282165-43e0-46b9-a3ff-b8c98d4443d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:25.561120 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 657/1324] 10.222.0.45 () {58 vars in 1062 bytes} [Thu Aug 4 02:09:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:09:25.571171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3f949-3213-4332-9934-7dc4a6a89eb7 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:09:25.572788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1cf3f949-3213-4332-9934-7dc4a6a89eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:25.593018 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:25.687916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3f949-3213-4332-9934-7dc4a6a89eb7 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:09:25.771801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01005f1d-b40b-43ce-a66a-26ee16caf553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:09:25.772413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01005f1d-b40b-43ce-a66a-26ee16caf553 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:25.772730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01005f1d-b40b-43ce-a66a-26ee16caf553 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:25.773005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01005f1d-b40b-43ce-a66a-26ee16caf553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:25.774726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-01005f1d-b40b-43ce-a66a-26ee16caf553 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:25.860065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:25.899916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01005f1d-b40b-43ce-a66a-26ee16caf553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:09:25.901410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01005f1d-b40b-43ce-a66a-26ee16caf553 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:09:25.920579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 668/1325] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:09:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:09:28.252268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449abe4-83f9-49d7-b175-69c115d046d1 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:28.252876 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449abe4-83f9-49d7-b175-69c115d046d1 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:28.252876 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449abe4-83f9-49d7-b175-69c115d046d1 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:28.254471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 658/1326] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:09:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:09:28.260165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41fa77-d1a9-4794-929c-43a307605dae None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:28.260713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41fa77-d1a9-4794-929c-43a307605dae None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:28.260812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe41fa77-d1a9-4794-929c-43a307605dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:28.334159 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-fe41fa77-d1a9-4794-929c-43a307605dae None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:09:28.335372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-fe41fa77-d1a9-4794-929c-43a307605dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:28.441008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 669/1327] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:09:28.478352 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449abe4-83f9-49d7-b175-69c115d046d1 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:09:28.479893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-1449abe4-83f9-49d7-b175-69c115d046d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:28.546656 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:28.598625 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449abe4-83f9-49d7-b175-69c115d046d1 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:09:28.676781 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5996da4-3875-4965-a5ca-0f79fc5f0f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:09:28.677401 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5996da4-3875-4965-a5ca-0f79fc5f0f88 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:28.677686 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5996da4-3875-4965-a5ca-0f79fc5f0f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:28.677960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5996da4-3875-4965-a5ca-0f79fc5f0f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:28.679613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d5996da4-3875-4965-a5ca-0f79fc5f0f88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:28.791854 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:28.794492 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5996da4-3875-4965-a5ca-0f79fc5f0f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:09:28.796585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5996da4-3875-4965-a5ca-0f79fc5f0f88 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:09:28.815927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 659/1328] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:09:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:09:30.943382 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf3b22-a9e6-42f4-a8e5-824bcaf937bd None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:30.943861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf3b22-a9e6-42f4-a8e5-824bcaf937bd None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:30.943952 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf3b22-a9e6-42f4-a8e5-824bcaf937bd None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:30.945539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 670/1329] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:09:30.951164 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00a53a-8637-4811-adc3-87768ba38bdb None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:30.951695 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00a53a-8637-4811-adc3-87768ba38bdb None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:30.951695 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00a53a-8637-4811-adc3-87768ba38bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:31.024443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-9f00a53a-8637-4811-adc3-87768ba38bdb None None] MFA Rules not processed for user `e88786550f274e61877748f90084c3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:09:31.025696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9f00a53a-8637-4811-adc3-87768ba38bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:31.152576 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 660/1330] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:09:30 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:09:31.191992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf3b22-a9e6-42f4-a8e5-824bcaf937bd None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:09:31.193600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-93bf3b22-a9e6-42f4-a8e5-824bcaf937bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:31.230221 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:31.307935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf3b22-a9e6-42f4-a8e5-824bcaf937bd None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:09:31.385292 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f50d58-5c9d-41ee-9f5c-a8f1c04bd9c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:09:31.385910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f50d58-5c9d-41ee-9f5c-a8f1c04bd9c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:31.386187 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f50d58-5c9d-41ee-9f5c-a8f1c04bd9c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:31.386455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f50d58-5c9d-41ee-9f5c-a8f1c04bd9c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:31.388080 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c6f50d58-5c9d-41ee-9f5c-a8f1c04bd9c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:31.467891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:31.505290 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f50d58-5c9d-41ee-9f5c-a8f1c04bd9c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:09:31.506724 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f50d58-5c9d-41ee-9f5c-a8f1c04bd9c8 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:09:31.526108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 671/1331] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:09:31 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:09:34.012529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e708e-c72f-45a4-9e80-f24ea2b5b585 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:34.013073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e708e-c72f-45a4-9e80-f24ea2b5b585 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:34.013073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e708e-c72f-45a4-9e80-f24ea2b5b585 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:34.014413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 661/1332] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:09:34.019497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220c993-4d9c-4551-ad63-e8b01be41805 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:34.019931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220c993-4d9c-4551-ad63-e8b01be41805 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:34.020048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8220c993-4d9c-4551-ad63-e8b01be41805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:34.084920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-8220c993-4d9c-4551-ad63-e8b01be41805 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:09:34.086118 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8220c993-4d9c-4551-ad63-e8b01be41805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:34.187375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 672/1333] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:09:34.227240 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e708e-c72f-45a4-9e80-f24ea2b5b585 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:09:34.227456 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7f9e708e-c72f-45a4-9e80-f24ea2b5b585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:34.298878 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:34.337537 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e708e-c72f-45a4-9e80-f24ea2b5b585 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:09:34.409705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f72a54-0b4e-43d2-a772-40786f5be503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:09:34.410317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f72a54-0b4e-43d2-a772-40786f5be503 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:34.410598 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f72a54-0b4e-43d2-a772-40786f5be503 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:34.410866 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f72a54-0b4e-43d2-a772-40786f5be503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:34.412492 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-27f72a54-0b4e-43d2-a772-40786f5be503 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:34.522495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f72a54-0b4e-43d2-a772-40786f5be503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:09:34.523940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f72a54-0b4e-43d2-a772-40786f5be503 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:09:34.534581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:34.543557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 662/1334] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:09:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:09:36.611245 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975edab-2f9d-4c91-9cf5-7b9b5f3a9b49 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:36.611882 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975edab-2f9d-4c91-9cf5-7b9b5f3a9b49 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:36.611882 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c975edab-2f9d-4c91-9cf5-7b9b5f3a9b49 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:36.613193 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 673/1335] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:09:36.618356 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b590f6-59ec-4f4a-a53b-cf6b242170a3 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:36.618811 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b590f6-59ec-4f4a-a53b-cf6b242170a3 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:36.618913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b590f6-59ec-4f4a-a53b-cf6b242170a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:36.685190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-41b590f6-59ec-4f4a-a53b-cf6b242170a3 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:09:36.686411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-41b590f6-59ec-4f4a-a53b-cf6b242170a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:36.782199 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 663/1336] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:09:36.822305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975edab-2f9d-4c91-9cf5-7b9b5f3a9b49 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:09:36.823829 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c975edab-2f9d-4c91-9cf5-7b9b5f3a9b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:36.864080 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:36.927945 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c975edab-2f9d-4c91-9cf5-7b9b5f3a9b49 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:09:37.001265 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ce017-3519-4fb2-b263-df0e718a99b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:09:37.001877 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ce017-3519-4fb2-b263-df0e718a99b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:37.002668 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ce017-3519-4fb2-b263-df0e718a99b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:37.002941 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ce017-3519-4fb2-b263-df0e718a99b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:37.004573 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e72ce017-3519-4fb2-b263-df0e718a99b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:37.087455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:37.113042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ce017-3519-4fb2-b263-df0e718a99b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:09:37.114524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ce017-3519-4fb2-b263-df0e718a99b6 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:09:37.132575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 674/1337] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:09:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:09:39.522596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aea5e8-69d4-43a0-9117-8fa762de9c9e None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:39.523123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aea5e8-69d4-43a0-9117-8fa762de9c9e None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:39.523123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aea5e8-69d4-43a0-9117-8fa762de9c9e None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:39.524494 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 664/1338] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:09:39.529971 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf967d-2ece-4d43-8c04-0f3b28a9b428 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:39.530448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf967d-2ece-4d43-8c04-0f3b28a9b428 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:39.530719 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cf967d-2ece-4d43-8c04-0f3b28a9b428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:39.603533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-75cf967d-2ece-4d43-8c04-0f3b28a9b428 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:09:39.604721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-75cf967d-2ece-4d43-8c04-0f3b28a9b428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:39.701017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 675/1339] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:09:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:09:39.738115 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aea5e8-69d4-43a0-9117-8fa762de9c9e None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:09:39.739632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-09aea5e8-69d4-43a0-9117-8fa762de9c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:39.813722 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:39.842692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aea5e8-69d4-43a0-9117-8fa762de9c9e None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:09:39.920853 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bd356-41cf-42bc-9ee1-e5d94d83de84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:09:39.921478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bd356-41cf-42bc-9ee1-e5d94d83de84 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:39.921755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bd356-41cf-42bc-9ee1-e5d94d83de84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:39.922027 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bd356-41cf-42bc-9ee1-e5d94d83de84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:39.923664 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-aa3bd356-41cf-42bc-9ee1-e5d94d83de84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:40.030782 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bd356-41cf-42bc-9ee1-e5d94d83de84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:09:40.032213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3bd356-41cf-42bc-9ee1-e5d94d83de84 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:09:40.049617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 665/1340] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:09:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:09:44.006568 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ee9db-9dba-48e2-9fba-f75e280fc0e2 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:44.007133 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ee9db-9dba-48e2-9fba-f75e280fc0e2 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:44.007269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ee9db-9dba-48e2-9fba-f75e280fc0e2 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:44.008744 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 676/1341] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:09:44.014166 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12b905-1c5d-4c1d-97a1-efa8ff9c2f93 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:44.014560 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12b905-1c5d-4c1d-97a1-efa8ff9c2f93 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:44.014717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12b905-1c5d-4c1d-97a1-efa8ff9c2f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:44.020757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:44.082445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-0d12b905-1c5d-4c1d-97a1-efa8ff9c2f93 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:09:44.083655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-0d12b905-1c5d-4c1d-97a1-efa8ff9c2f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:44.181462 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 666/1342] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:09:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:09:44.221448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ee9db-9dba-48e2-9fba-f75e280fc0e2 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:09:44.223261 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-159ee9db-9dba-48e2-9fba-f75e280fc0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:44.274086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:44.332351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ee9db-9dba-48e2-9fba-f75e280fc0e2 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:09:44.414404 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1545d2f-3963-4b8a-b3b5-d3cc728efd94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:09:44.415025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1545d2f-3963-4b8a-b3b5-d3cc728efd94 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:44.415303 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1545d2f-3963-4b8a-b3b5-d3cc728efd94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:44.415575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1545d2f-3963-4b8a-b3b5-d3cc728efd94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:44.417214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d1545d2f-3963-4b8a-b3b5-d3cc728efd94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:44.514285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:44.533791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1545d2f-3963-4b8a-b3b5-d3cc728efd94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:09:44.535267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1545d2f-3963-4b8a-b3b5-d3cc728efd94 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:09:44.553980 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 677/1343] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:09:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:09:47.402724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e1eda-4898-4c08-befa-010af3657128 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:47.403287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e1eda-4898-4c08-befa-010af3657128 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:47.403287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e1eda-4898-4c08-befa-010af3657128 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:47.404673 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 667/1344] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:09:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:09:47.410697 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc20ae-d942-4c06-a14d-dce962cc6f3a None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:47.411149 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc20ae-d942-4c06-a14d-dce962cc6f3a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:47.411239 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc20ae-d942-4c06-a14d-dce962cc6f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:47.477584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-d9fc20ae-d942-4c06-a14d-dce962cc6f3a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:09:47.478809 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d9fc20ae-d942-4c06-a14d-dce962cc6f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:47.582176 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 678/1345] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:09:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:09:47.622807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e1eda-4898-4c08-befa-010af3657128 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:09:47.624335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-1f8e1eda-4898-4c08-befa-010af3657128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:47.717674 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:47.735439 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e1eda-4898-4c08-befa-010af3657128 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:09:47.808294 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db4ef1-8c0d-4888-ba5b-3994a06e16db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:09:47.808964 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db4ef1-8c0d-4888-ba5b-3994a06e16db service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:47.809243 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db4ef1-8c0d-4888-ba5b-3994a06e16db service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:47.809519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db4ef1-8c0d-4888-ba5b-3994a06e16db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:47.811155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e1db4ef1-8c0d-4888-ba5b-3994a06e16db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:47.923254 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db4ef1-8c0d-4888-ba5b-3994a06e16db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:09:47.924872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db4ef1-8c0d-4888-ba5b-3994a06e16db service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:09:47.942864 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 668/1346] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:09:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:09:52.157359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd6104-5ce5-453f-a83f-03e958f147ce None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:52.157833 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd6104-5ce5-453f-a83f-03e958f147ce None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:52.157924 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cd6104-5ce5-453f-a83f-03e958f147ce None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:52.159954 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 679/1347] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:09:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:09:52.165787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f80ee-4c62-487e-b9e6-02d3d499d04c None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:52.166208 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f80ee-4c62-487e-b9e6-02d3d499d04c None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:52.166358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134f80ee-4c62-487e-b9e6-02d3d499d04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:52.183553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:52.253345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-134f80ee-4c62-487e-b9e6-02d3d499d04c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:09:52.254564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-134f80ee-4c62-487e-b9e6-02d3d499d04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:52.371522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 669/1348] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:09:52.410748 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd6104-5ce5-453f-a83f-03e958f147ce None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:09:52.412084 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b4cd6104-5ce5-453f-a83f-03e958f147ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:52.482774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:52.535622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cd6104-5ce5-453f-a83f-03e958f147ce None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:09:52.629386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ea296-a961-40a0-8d33-51bafeaee4c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:09:52.630054 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ea296-a961-40a0-8d33-51bafeaee4c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:52.630333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ea296-a961-40a0-8d33-51bafeaee4c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:52.630607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ea296-a961-40a0-8d33-51bafeaee4c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:52.632269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-de1ea296-a961-40a0-8d33-51bafeaee4c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:52.747839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:09:52.753041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ea296-a961-40a0-8d33-51bafeaee4c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:09:52.754506 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1ea296-a961-40a0-8d33-51bafeaee4c7 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:09:52.774424 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 680/1349] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:09:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:09:57.682567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e71f5-273b-4f76-9dd6-da624e17e6ff None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:57.683991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e71f5-273b-4f76-9dd6-da624e17e6ff None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:57.684487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e71f5-273b-4f76-9dd6-da624e17e6ff None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:57.686131 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 670/1350] 10.222.0.45 () {54 vars in 921 bytes} [Thu Aug 4 02:09:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:09:57.700964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8014c2-9f7c-492a-81d5-f5846310f1a9 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:09:57.703758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8014c2-9f7c-492a-81d5-f5846310f1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:09:57.706655 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8014c2-9f7c-492a-81d5-f5846310f1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:09:57.834661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-ad8014c2-9f7c-492a-81d5-f5846310f1a9 None None] MFA Rules not processed for user `796772cf491047c8b0a609dae9402307`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:09:57.835861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-ad8014c2-9f7c-492a-81d5-f5846310f1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:09:57.969843 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 681/1351] 10.222.0.45 () {58 vars in 1018 bytes} [Thu Aug 4 02:09:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:02.104209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53ac4d-b2b5-434c-a081-39c543f916de None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:02.104943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53ac4d-b2b5-434c-a081-39c543f916de None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:02.104943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53ac4d-b2b5-434c-a081-39c543f916de None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:02.106042 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 671/1352] 10.222.0.45 () {54 vars in 921 bytes} [Thu Aug 4 02:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:02.111921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985a3e3-bea6-43a2-8c47-919408ebec92 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:02.112399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985a3e3-bea6-43a2-8c47-919408ebec92 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:02.112496 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985a3e3-bea6-43a2-8c47-919408ebec92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:02.172928 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:02.199187 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-d985a3e3-bea6-43a2-8c47-919408ebec92 None None] MFA Rules not processed for user `796772cf491047c8b0a609dae9402307`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:02.200401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d985a3e3-bea6-43a2-8c47-919408ebec92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:02.312610 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 682/1353] 10.222.0.45 () {58 vars in 1018 bytes} [Thu Aug 4 02:10:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:02.369534 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=59 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:03.764499 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a25e7-aea6-4071-b178-53cac9c59c75 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:03.765145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a25e7-aea6-4071-b178-53cac9c59c75 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:03.765145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a25e7-aea6-4071-b178-53cac9c59c75 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:03.766436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 672/1354] 10.222.0.45 () {54 vars in 921 bytes} [Thu Aug 4 02:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:03.771472 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227462df-8857-4c88-94cc-aeda93eee197 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:03.771891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227462df-8857-4c88-94cc-aeda93eee197 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:03.772170 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227462df-8857-4c88-94cc-aeda93eee197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:03.848028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-227462df-8857-4c88-94cc-aeda93eee197 None None] MFA Rules not processed for user `796772cf491047c8b0a609dae9402307`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:03.849252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-227462df-8857-4c88-94cc-aeda93eee197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:03.893506 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:03.968758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 683/1355] 10.222.0.45 () {58 vars in 1018 bytes} [Thu Aug 4 02:10:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:07.401355 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914f276-353e-4d96-ba6f-fabf14eff60d None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:07.403406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914f276-353e-4d96-ba6f-fabf14eff60d None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:07.404188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914f276-353e-4d96-ba6f-fabf14eff60d None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:07.407344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 673/1356] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:10:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:07.413832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e4ca5-83d6-4d1f-ae8f-eaadf29a61eb None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:07.414778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e4ca5-83d6-4d1f-ae8f-eaadf29a61eb None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:07.415231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e4ca5-83d6-4d1f-ae8f-eaadf29a61eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:07.497754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-3e6e4ca5-83d6-4d1f-ae8f-eaadf29a61eb None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:07.499418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-3e6e4ca5-83d6-4d1f-ae8f-eaadf29a61eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:07.615905 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 684/1357] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:07.669570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abce0c6-a199-47a9-b9fe-bfb8810ce8b3 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:07.670126 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abce0c6-a199-47a9-b9fe-bfb8810ce8b3 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:07.670759 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abce0c6-a199-47a9-b9fe-bfb8810ce8b3 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:07.672798 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 674/1358] 10.222.0.45 () {54 vars in 962 bytes} [Thu Aug 4 02:10:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:07.682517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd210d-db21-4254-9ab9-9c682da56a03 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:07.683385 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd210d-db21-4254-9ab9-9c682da56a03 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:07.684196 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd210d-db21-4254-9ab9-9c682da56a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:07.709605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:07.776219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c2dd210d-db21-4254-9ab9-9c682da56a03 None None] MFA Rules not processed for user `c571406d2a3f40dab035a918399d6f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:07.777441 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c2dd210d-db21-4254-9ab9-9c682da56a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:07.896476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 685/1359] 10.222.0.45 () {58 vars in 1059 bytes} [Thu Aug 4 02:10:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:10:07.904118 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abce0c6-a199-47a9-b9fe-bfb8810ce8b3 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:10:07.905667 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-8abce0c6-a199-47a9-b9fe-bfb8810ce8b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:08.074010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abce0c6-a199-47a9-b9fe-bfb8810ce8b3 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:10:08.153774 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:08.202699 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4699a79a-ac8a-4d6e-a895-ef1a286fe0f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:10:08.203628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699a79a-ac8a-4d6e-a895-ef1a286fe0f4 service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:08.204215 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699a79a-ac8a-4d6e-a895-ef1a286fe0f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:08.205008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4699a79a-ac8a-4d6e-a895-ef1a286fe0f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:08.208288 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-4699a79a-ac8a-4d6e-a895-ef1a286fe0f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:08.318646 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699a79a-ac8a-4d6e-a895-ef1a286fe0f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:10:08.320685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4699a79a-ac8a-4d6e-a895-ef1a286fe0f4 service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:10:08.340119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 675/1360] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:10:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:10:10.582125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7139e9c-15d9-4f7c-8b51-874b8117d38e None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:10.582125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7139e9c-15d9-4f7c-8b51-874b8117d38e None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:10.582125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7139e9c-15d9-4f7c-8b51-874b8117d38e None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:10.584372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 686/1361] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:10:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:10.590144 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae51d1d-2534-4493-8f3a-439b7c979b65 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:10.590606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae51d1d-2534-4493-8f3a-439b7c979b65 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:10.590717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae51d1d-2534-4493-8f3a-439b7c979b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:10.680182 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-3ae51d1d-2534-4493-8f3a-439b7c979b65 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:10.682077 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-3ae51d1d-2534-4493-8f3a-439b7c979b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:10.723449 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:10.787028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 676/1362] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:10.832445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7139e9c-15d9-4f7c-8b51-874b8117d38e None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:10:10.834346 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e7139e9c-15d9-4f7c-8b51-874b8117d38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:10.910664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:11.014999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7139e9c-15d9-4f7c-8b51-874b8117d38e None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:10:11.167820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dd3d90-09ce-4bce-a0c3-9e8fde9f6e90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:10:11.168710 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd3d90-09ce-4bce-a0c3-9e8fde9f6e90 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:11.169077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd3d90-09ce-4bce-a0c3-9e8fde9f6e90 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:11.169476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dd3d90-09ce-4bce-a0c3-9e8fde9f6e90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:11.171216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f4dd3d90-09ce-4bce-a0c3-9e8fde9f6e90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:11.197647 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:11.269837 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd3d90-09ce-4bce-a0c3-9e8fde9f6e90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:10:11.272391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dd3d90-09ce-4bce-a0c3-9e8fde9f6e90 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:10:11.299072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 687/1363] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:10:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:10:13.677285 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa95e0b-4ed0-4978-ae6d-91285aff3db7 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:13.678207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa95e0b-4ed0-4978-ae6d-91285aff3db7 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:13.678488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa95e0b-4ed0-4978-ae6d-91285aff3db7 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:13.680175 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 677/1364] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:10:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:13.687374 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba2d7b-b272-4be6-9f7f-b0a2eab869f8 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:13.688042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba2d7b-b272-4be6-9f7f-b0a2eab869f8 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:13.688667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba2d7b-b272-4be6-9f7f-b0a2eab869f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:13.768225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-9eba2d7b-b272-4be6-9f7f-b0a2eab869f8 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:13.769834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9eba2d7b-b272-4be6-9f7f-b0a2eab869f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:13.885287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:13.892799 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 688/1365] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:10:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:13.935254 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d506144-4c93-45b8-af99-1c3efbd62bd3 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:13.935805 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d506144-4c93-45b8-af99-1c3efbd62bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:13.936289 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d506144-4c93-45b8-af99-1c3efbd62bd3 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:13.937851 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 678/1366] 10.222.0.45 () {54 vars in 962 bytes} [Thu Aug 4 02:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:13.944394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3d21d-8fe4-4bd2-a663-2f4f7f453827 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:13.944951 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3d21d-8fe4-4bd2-a663-2f4f7f453827 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:13.945238 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3d21d-8fe4-4bd2-a663-2f4f7f453827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:14.030934 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-bcd3d21d-8fe4-4bd2-a663-2f4f7f453827 None None] MFA Rules not processed for user `c571406d2a3f40dab035a918399d6f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:14.032147 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-bcd3d21d-8fe4-4bd2-a663-2f4f7f453827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:14.155545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 689/1367] 10.222.0.45 () {58 vars in 1059 bytes} [Thu Aug 4 02:10:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:14.163140 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d506144-4c93-45b8-af99-1c3efbd62bd3 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:10:14.164677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-5d506144-4c93-45b8-af99-1c3efbd62bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:14.395543 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d506144-4c93-45b8-af99-1c3efbd62bd3 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:10:14.424998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:14.537859 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593ad33-e67a-4623-b630-083d5ed9b7f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:10:14.538545 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593ad33-e67a-4623-b630-083d5ed9b7f6 service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:14.538940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593ad33-e67a-4623-b630-083d5ed9b7f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:14.539299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593ad33-e67a-4623-b630-083d5ed9b7f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:14.541058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e593ad33-e67a-4623-b630-083d5ed9b7f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:14.637850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593ad33-e67a-4623-b630-083d5ed9b7f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:10:14.639646 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e593ad33-e67a-4623-b630-083d5ed9b7f6 service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:10:14.666289 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 679/1368] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:10:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:10:15.619576 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bd992-fb94-4624-8f08-36d6e34748ef None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:15.620152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bd992-fb94-4624-8f08-36d6e34748ef None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:15.620152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bd992-fb94-4624-8f08-36d6e34748ef None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:15.621672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 690/1369] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:15.627159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499d151-fde0-4a68-a4b5-697dca7bea0b None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:15.627594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499d151-fde0-4a68-a4b5-697dca7bea0b None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:15.627791 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f499d151-fde0-4a68-a4b5-697dca7bea0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:15.679561 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:15.720011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-f499d151-fde0-4a68-a4b5-697dca7bea0b None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:15.721275 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-f499d151-fde0-4a68-a4b5-697dca7bea0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:15.864682 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 680/1370] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:10:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:16.232293 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31281457-da0b-46ab-bc01-e625fa29d5ef None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:16.232844 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31281457-da0b-46ab-bc01-e625fa29d5ef None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:16.234206 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31281457-da0b-46ab-bc01-e625fa29d5ef None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:16.236605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 691/1371] 10.222.0.45 () {54 vars in 964 bytes} [Thu Aug 4 02:10:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:16.242393 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a848a08-3a55-448e-b3e1-682ab8ca7fe9 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:16.242677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a848a08-3a55-448e-b3e1-682ab8ca7fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:16.242960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a848a08-3a55-448e-b3e1-682ab8ca7fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:16.336968 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-2a848a08-3a55-448e-b3e1-682ab8ca7fe9 None None] MFA Rules not processed for user `f4b8838129174fbb8cdd32bd5d4d6101`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:16.342646 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2a848a08-3a55-448e-b3e1-682ab8ca7fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:16.395134 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:16.429887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 681/1372] 10.222.0.45 () {58 vars in 1061 bytes} [Thu Aug 4 02:10:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:16.437741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31281457-da0b-46ab-bc01-e625fa29d5ef None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:10:16.439541 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-31281457-da0b-46ab-bc01-e625fa29d5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:16.506439 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:16.572669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31281457-da0b-46ab-bc01-e625fa29d5ef None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:10:16.683860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e338d-d5ba-49bc-9fa8-070663a80381 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:10:16.685614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e338d-d5ba-49bc-9fa8-070663a80381 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:16.686165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e338d-d5ba-49bc-9fa8-070663a80381 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:16.686686 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e338d-d5ba-49bc-9fa8-070663a80381 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:16.689883 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-503e338d-d5ba-49bc-9fa8-070663a80381 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:16.810294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:16.828075 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e0d95-6fe6-48ed-927e-92432b9974cb None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:16.828566 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e0d95-6fe6-48ed-927e-92432b9974cb None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:16.828688 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e0d95-6fe6-48ed-927e-92432b9974cb None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:16.828913 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e338d-d5ba-49bc-9fa8-070663a80381 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:10:16.831600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e338d-d5ba-49bc-9fa8-070663a80381 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:10:16.831743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 682/1373] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:10:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:10:16.838611 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0feb65-dd19-409b-9da4-4d6aae994357 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:16.838988 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0feb65-dd19-409b-9da4-4d6aae994357 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:16.839252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0feb65-dd19-409b-9da4-4d6aae994357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:16.854247 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 692/1374] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:10:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:10:16.956899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-bd0feb65-dd19-409b-9da4-4d6aae994357 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:16.959611 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-bd0feb65-dd19-409b-9da4-4d6aae994357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:17.173716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 683/1375] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:10:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:10:17.217036 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e338d-d5ba-49bc-9fa8-070663a80381 service cinder] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:10:17.218855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-503e338d-d5ba-49bc-9fa8-070663a80381 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:17.341821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e338d-d5ba-49bc-9fa8-070663a80381 service cinder] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:10:17.448785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:17.461867 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae28f79-e0c1-404d-ad9f-beba65773f9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:10:17.462486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae28f79-e0c1-404d-ad9f-beba65773f9b service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:17.462768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae28f79-e0c1-404d-ad9f-beba65773f9b service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:17.463041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae28f79-e0c1-404d-ad9f-beba65773f9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:17.464702 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7ae28f79-e0c1-404d-ad9f-beba65773f9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:17.558602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae28f79-e0c1-404d-ad9f-beba65773f9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:10:17.560262 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae28f79-e0c1-404d-ad9f-beba65773f9b service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:10:17.579800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 693/1376] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:10:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:10:19.628514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f2d1e-67c4-43a3-9029-822daf1ec941 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:19.629118 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f2d1e-67c4-43a3-9029-822daf1ec941 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:19.629118 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f2d1e-67c4-43a3-9029-822daf1ec941 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:19.630462 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 684/1377] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:19.635594 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e8d84-ab36-47b0-8cc0-7327682eae7c None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:19.636031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e8d84-ab36-47b0-8cc0-7327682eae7c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:19.636188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e8d84-ab36-47b0-8cc0-7327682eae7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:19.706268 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-109e8d84-ab36-47b0-8cc0-7327682eae7c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:19.708079 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-109e8d84-ab36-47b0-8cc0-7327682eae7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:19.769081 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:19.821099 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 694/1378] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:10:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:19.855492 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f2d1e-67c4-43a3-9029-822daf1ec941 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:10:19.857053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d68f2d1e-67c4-43a3-9029-822daf1ec941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:19.932004 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:20.027915 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f2d1e-67c4-43a3-9029-822daf1ec941 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:10:20.140431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626f943-26fc-4cd7-9e58-7fd0a0b58a95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:10:20.141182 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626f943-26fc-4cd7-9e58-7fd0a0b58a95 service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:20.141461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626f943-26fc-4cd7-9e58-7fd0a0b58a95 service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:20.141734 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626f943-26fc-4cd7-9e58-7fd0a0b58a95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:20.143873 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9626f943-26fc-4cd7-9e58-7fd0a0b58a95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:20.159785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:20.239128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9626f943-26fc-4cd7-9e58-7fd0a0b58a95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:10:20.241159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9626f943-26fc-4cd7-9e58-7fd0a0b58a95 service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:10:20.261348 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 685/1379] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:10:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:10:22.380937 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa5930-920a-4b81-8991-f6bf148aaed2 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:22.381477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa5930-920a-4b81-8991-f6bf148aaed2 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:22.381477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa5930-920a-4b81-8991-f6bf148aaed2 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:22.383018 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 695/1380] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:22.388112 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4f1c1-dc77-47ce-91ba-e525b2affbee None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:22.388560 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4f1c1-dc77-47ce-91ba-e525b2affbee None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:22.388696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4f1c1-dc77-47ce-91ba-e525b2affbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:22.469634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-2ea4f1c1-dc77-47ce-91ba-e525b2affbee None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:22.471096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2ea4f1c1-dc77-47ce-91ba-e525b2affbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:22.581036 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:22.600687 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 686/1381] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:10:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:22.643693 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa5930-920a-4b81-8991-f6bf148aaed2 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:10:22.645212 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b0aa5930-920a-4b81-8991-f6bf148aaed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:22.813796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa5930-920a-4b81-8991-f6bf148aaed2 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:10:22.867948 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:22.938187 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834a471-ce84-40e6-b727-ff652c0eab86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:10:22.938725 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834a471-ce84-40e6-b727-ff652c0eab86 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:22.939004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834a471-ce84-40e6-b727-ff652c0eab86 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:22.939274 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834a471-ce84-40e6-b727-ff652c0eab86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:22.940928 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c834a471-ce84-40e6-b727-ff652c0eab86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:23.036081 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834a471-ce84-40e6-b727-ff652c0eab86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:10:23.037694 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834a471-ce84-40e6-b727-ff652c0eab86 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:10:23.056555 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 696/1382] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:10:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:10:25.315086 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c7781-2628-4ab0-9de9-3cf5e0fe07ec None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:25.315798 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c7781-2628-4ab0-9de9-3cf5e0fe07ec None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:25.315798 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c7781-2628-4ab0-9de9-3cf5e0fe07ec None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:25.317050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 687/1383] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:25.322964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5aaac-351e-4cb4-b42c-17e3ed2eba9b None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:25.323516 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5aaac-351e-4cb4-b42c-17e3ed2eba9b None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:25.323675 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5aaac-351e-4cb4-b42c-17e3ed2eba9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:25.395297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c2c5aaac-351e-4cb4-b42c-17e3ed2eba9b None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:25.396889 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c2c5aaac-351e-4cb4-b42c-17e3ed2eba9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:25.399508 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:25.528755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 697/1384] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:25.575950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c7781-2628-4ab0-9de9-3cf5e0fe07ec None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:10:25.577474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-457c7781-2628-4ab0-9de9-3cf5e0fe07ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:25.751177 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c7781-2628-4ab0-9de9-3cf5e0fe07ec None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:10:25.828057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:25.891276 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46990eff-0197-4423-bc5e-3833cf72a673 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:10:25.892087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46990eff-0197-4423-bc5e-3833cf72a673 service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:25.892495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46990eff-0197-4423-bc5e-3833cf72a673 service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:25.892973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46990eff-0197-4423-bc5e-3833cf72a673 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:25.895154 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-46990eff-0197-4423-bc5e-3833cf72a673 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:26.004552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46990eff-0197-4423-bc5e-3833cf72a673 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:10:26.006155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46990eff-0197-4423-bc5e-3833cf72a673 service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:10:26.024864 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 688/1385] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:10:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:10:28.117375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635bdca-d385-43fc-9a67-25dfd3709305 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:28.117961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635bdca-d385-43fc-9a67-25dfd3709305 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:28.118328 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635bdca-d385-43fc-9a67-25dfd3709305 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:28.120540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 698/1386] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:10:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:28.126556 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c13f4-935f-418c-8add-7fa1ba0fa0a3 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:28.127017 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c13f4-935f-418c-8add-7fa1ba0fa0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:28.127104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c13f4-935f-418c-8add-7fa1ba0fa0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:28.196117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-248c13f4-935f-418c-8add-7fa1ba0fa0a3 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:28.197423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-248c13f4-935f-418c-8add-7fa1ba0fa0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:28.232513 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:28.329586 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 689/1387] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:10:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:28.377805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635bdca-d385-43fc-9a67-25dfd3709305 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:10:28.380287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7635bdca-d385-43fc-9a67-25dfd3709305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:28.522053 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:28.540839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635bdca-d385-43fc-9a67-25dfd3709305 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:10:28.699146 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c82a78-0984-4ad2-ab72-fbc1ef6bd49d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:10:28.699799 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c82a78-0984-4ad2-ab72-fbc1ef6bd49d service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:28.700077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c82a78-0984-4ad2-ab72-fbc1ef6bd49d service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:28.700359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c82a78-0984-4ad2-ab72-fbc1ef6bd49d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:28.702107 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-67c82a78-0984-4ad2-ab72-fbc1ef6bd49d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:28.801582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c82a78-0984-4ad2-ab72-fbc1ef6bd49d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:10:28.803232 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c82a78-0984-4ad2-ab72-fbc1ef6bd49d service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:10:28.808221 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:28.823739 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 699/1388] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:10:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:10:31.189261 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfd7cc-69c6-40e1-92f0-696ae38f24d6 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:31.189935 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfd7cc-69c6-40e1-92f0-696ae38f24d6 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:31.189935 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfd7cc-69c6-40e1-92f0-696ae38f24d6 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:31.191296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 690/1389] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:31.197328 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d0d53-6d8e-4048-831c-d13027e1f360 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:31.197783 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d0d53-6d8e-4048-831c-d13027e1f360 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:31.197890 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d0d53-6d8e-4048-831c-d13027e1f360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:31.264099 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-e35d0d53-6d8e-4048-831c-d13027e1f360 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:31.265349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e35d0d53-6d8e-4048-831c-d13027e1f360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:31.363314 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 700/1390] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:31.408025 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfd7cc-69c6-40e1-92f0-696ae38f24d6 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:10:31.409620 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-8dcfd7cc-69c6-40e1-92f0-696ae38f24d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:31.558011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfd7cc-69c6-40e1-92f0-696ae38f24d6 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:10:31.566928 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:31.676394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bac770-b880-49f2-98b2-604bcfe97dac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:10:31.677043 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bac770-b880-49f2-98b2-604bcfe97dac service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:31.677423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bac770-b880-49f2-98b2-604bcfe97dac service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:31.677700 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bac770-b880-49f2-98b2-604bcfe97dac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:31.679338 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-94bac770-b880-49f2-98b2-604bcfe97dac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:31.771190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bac770-b880-49f2-98b2-604bcfe97dac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:10:31.772833 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bac770-b880-49f2-98b2-604bcfe97dac service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:10:31.791453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 691/1391] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:10:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:10:32.089938 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358a9d4-1b2d-463d-9c5f-5f908e1cae5e None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:32.090147 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358a9d4-1b2d-463d-9c5f-5f908e1cae5e None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:32.090423 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358a9d4-1b2d-463d-9c5f-5f908e1cae5e None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:32.091857 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 701/1392] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:32.097331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9b5f8-6831-4b27-9e61-532b033238b9 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:32.097615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9b5f8-6831-4b27-9e61-532b033238b9 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:32.097888 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd9b5f8-6831-4b27-9e61-532b033238b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:32.113899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:32.166112 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-bfd9b5f8-6831-4b27-9e61-532b033238b9 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:32.167290 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-bfd9b5f8-6831-4b27-9e61-532b033238b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:32.264663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 692/1393] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:32.322462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb818f-38fb-420a-ba9b-be57654f9deb None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:32.323034 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb818f-38fb-420a-ba9b-be57654f9deb None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:32.323034 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edb818f-38fb-420a-ba9b-be57654f9deb None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:32.324340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 702/1394] 10.222.0.45 () {54 vars in 966 bytes} [Thu Aug 4 02:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:32.329480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193c340-678e-49d0-9dc5-7164504fe229 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:32.329770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193c340-678e-49d0-9dc5-7164504fe229 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:32.330069 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2193c340-678e-49d0-9dc5-7164504fe229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:32.408694 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-2193c340-678e-49d0-9dc5-7164504fe229 None None] MFA Rules not processed for user `14ff31d52ecf4c7d93a65bbfca7285c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:32.409927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2193c340-678e-49d0-9dc5-7164504fe229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:32.443937 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:32.486525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 693/1395] 10.222.0.45 () {58 vars in 1063 bytes} [Thu Aug 4 02:10:32 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:32.494200 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb818f-38fb-420a-ba9b-be57654f9deb None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:10:32.495734 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-2edb818f-38fb-420a-ba9b-be57654f9deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:32.573137 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:32.605672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edb818f-38fb-420a-ba9b-be57654f9deb None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:10:32.683018 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d669e3d-03d2-496b-9354-6a660643edbc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ff31d52ecf4c7d93a65bbfca7285c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:10:32.683563 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d669e3d-03d2-496b-9354-6a660643edbc service glance] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:32.683829 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d669e3d-03d2-496b-9354-6a660643edbc service glance] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:32.684095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d669e3d-03d2-496b-9354-6a660643edbc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:32.685751 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4d669e3d-03d2-496b-9354-6a660643edbc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:32.794460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d669e3d-03d2-496b-9354-6a660643edbc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:10:32.795901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d669e3d-03d2-496b-9354-6a660643edbc service glance] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:10:32.812081 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:32.815462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 703/1396] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:10:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:10:33.569640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f39b8-9b20-45d0-b96d-e50d350206aa None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:33.570388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f39b8-9b20-45d0-b96d-e50d350206aa None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:33.570500 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34f39b8-9b20-45d0-b96d-e50d350206aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:33.638007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-b34f39b8-9b20-45d0-b96d-e50d350206aa None None] MFA Rules not processed for user `e54e713b52e84393b018807f8153922e`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:33.639213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-b34f39b8-9b20-45d0-b96d-e50d350206aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:33.734786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 694/1397] 10.222.0.45 () {58 vars in 1009 bytes} [Thu Aug 4 02:10:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:33.776975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6032099-32a3-4ee2-8182-f6fb4590926e None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:33.777484 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6032099-32a3-4ee2-8182-f6fb4590926e None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:33.777654 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6032099-32a3-4ee2-8182-f6fb4590926e None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:33.779020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 704/1398] 10.222.0.45 () {54 vars in 943 bytes} [Thu Aug 4 02:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:33.785785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73b768-2dad-4907-a331-60624aa990be None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:33.786067 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73b768-2dad-4907-a331-60624aa990be None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:33.786337 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73b768-2dad-4907-a331-60624aa990be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:33.844871 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:33.865845 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-aa73b768-2dad-4907-a331-60624aa990be None None] MFA Rules not processed for user `155a7276efab426cb34175683c475f33`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:33.867199 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-aa73b768-2dad-4907-a331-60624aa990be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:33.880142 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6e208-edc0-4dd7-a17f-9b2c2f350c07 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:33.880428 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6e208-edc0-4dd7-a17f-9b2c2f350c07 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:33.880749 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6e208-edc0-4dd7-a17f-9b2c2f350c07 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:33.882098 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 705/1399] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:10:33.887185 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fdc23-eaf9-49a4-98ee-b8c77baa883a None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:33.887469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fdc23-eaf9-49a4-98ee-b8c77baa883a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:33.887741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451fdc23-eaf9-49a4-98ee-b8c77baa883a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:33.960258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-451fdc23-eaf9-49a4-98ee-b8c77baa883a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:33.961486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-451fdc23-eaf9-49a4-98ee-b8c77baa883a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:33.976239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 695/1400] 10.222.0.45 () {58 vars in 1040 bytes} [Thu Aug 4 02:10:33 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:33.985886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73b768-2dad-4907-a331-60624aa990be None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:10:33.996142 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73b768-2dad-4907-a331-60624aa990be None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:10:34.060111 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 706/1401] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:10:34.095971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:34.101538 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844e87e7-7d14-4deb-b3a1-e5803ceec05e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '155a7276efab426cb34175683c475f33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:10:34.102387 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e87e7-7d14-4deb-b3a1-e5803ceec05e service swift] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:34.102844 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e87e7-7d14-4deb-b3a1-e5803ceec05e service swift] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:34.103119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844e87e7-7d14-4deb-b3a1-e5803ceec05e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:34.105247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-844e87e7-7d14-4deb-b3a1-e5803ceec05e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:34.105912 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fdc23-eaf9-49a4-98ee-b8c77baa883a None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:10:34.107074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-451fdc23-eaf9-49a4-98ee-b8c77baa883a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:34.183264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:34.191956 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e87e7-7d14-4deb-b3a1-e5803ceec05e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:10:34.193825 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844e87e7-7d14-4deb-b3a1-e5803ceec05e service swift] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:10:34.202378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 696/1402] 10.222.0.45 () {60 vars in 1384 bytes} [Thu Aug 4 02:10:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 02:10:34.224103 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451fdc23-eaf9-49a4-98ee-b8c77baa883a None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:10:34.370504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f6502-08be-4970-a875-3e2d636c8310 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:10:34.373503 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f6502-08be-4970-a875-3e2d636c8310 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:34.373853 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f6502-08be-4970-a875-3e2d636c8310 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:34.374194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f6502-08be-4970-a875-3e2d636c8310 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:34.375999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d24f6502-08be-4970-a875-3e2d636c8310 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:34.470826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:34.490134 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f6502-08be-4970-a875-3e2d636c8310 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:10:34.491730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f6502-08be-4970-a875-3e2d636c8310 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:10:34.515751 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 707/1403] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:10:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:10:36.657555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba354e-a432-42be-965a-12f689be9faa None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:36.657555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba354e-a432-42be-965a-12f689be9faa None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:36.657555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba354e-a432-42be-965a-12f689be9faa None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:36.658917 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 697/1404] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:10:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:36.670223 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211f93-0415-4f35-804e-deef90ddd697 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:36.670685 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211f93-0415-4f35-804e-deef90ddd697 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:36.670768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8211f93-0415-4f35-804e-deef90ddd697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:36.741770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-b8211f93-0415-4f35-804e-deef90ddd697 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:36.743221 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b8211f93-0415-4f35-804e-deef90ddd697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:36.856183 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 708/1405] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:36.905394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba354e-a432-42be-965a-12f689be9faa None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:10:36.905394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2cba354e-a432-42be-965a-12f689be9faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:37.059333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:37.072429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba354e-a432-42be-965a-12f689be9faa None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:10:37.199855 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cda540-bd79-428a-8550-1d7f6d7b89e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:10:37.200477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cda540-bd79-428a-8550-1d7f6d7b89e3 service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:37.200788 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cda540-bd79-428a-8550-1d7f6d7b89e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:37.201070 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cda540-bd79-428a-8550-1d7f6d7b89e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:37.202739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-42cda540-bd79-428a-8550-1d7f6d7b89e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:37.303918 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cda540-bd79-428a-8550-1d7f6d7b89e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:10:37.305573 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cda540-bd79-428a-8550-1d7f6d7b89e3 service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:10:37.317712 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:37.328927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 698/1406] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:10:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:10:39.451100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7b2b-d95e-4658-993a-a3000226f39e None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:39.451667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7b2b-d95e-4658-993a-a3000226f39e None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:39.451667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7b2b-d95e-4658-993a-a3000226f39e None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:39.453166 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 709/1407] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:39.458987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45757d2-436c-42e5-90f1-32f5d472936f None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:39.459459 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45757d2-436c-42e5-90f1-32f5d472936f None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:39.459552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45757d2-436c-42e5-90f1-32f5d472936f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:39.534277 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-a45757d2-436c-42e5-90f1-32f5d472936f None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:39.536222 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a45757d2-436c-42e5-90f1-32f5d472936f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:39.648098 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 699/1408] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:10:39.693795 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c7b2b-d95e-4658-993a-a3000226f39e None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:10:39.695579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-6b0c7b2b-d95e-4658-993a-a3000226f39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:39.756483 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:39.858300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c7b2b-d95e-4658-993a-a3000226f39e None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:10:39.998288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f9fb11-f068-43e3-ae07-9981b30541dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:10:39.998814 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9fb11-f068-43e3-ae07-9981b30541dd service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:39.999090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9fb11-f068-43e3-ae07-9981b30541dd service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:39.999361 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f9fb11-f068-43e3-ae07-9981b30541dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:40.001039 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e4f9fb11-f068-43e3-ae07-9981b30541dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:40.014456 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:40.123894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9fb11-f068-43e3-ae07-9981b30541dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:10:40.125587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f9fb11-f068-43e3-ae07-9981b30541dd service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:10:40.145743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 710/1409] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:10:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:10:49.646250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=67 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:50.143179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=39 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:56.733886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce5a6b-15ee-4322-86a7-254f4bfbb932 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:56.734442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce5a6b-15ee-4322-86a7-254f4bfbb932 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:56.734442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ce5a6b-15ee-4322-86a7-254f4bfbb932 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:56.736477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 700/1410] 10.222.0.45 () {54 vars in 918 bytes} [Thu Aug 4 02:10:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:56.741629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ca78f-62eb-47f3-aff3-a33543612a36 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:56.742055 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ca78f-62eb-47f3-aff3-a33543612a36 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:56.742158 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298ca78f-62eb-47f3-aff3-a33543612a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:56.829375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-298ca78f-62eb-47f3-aff3-a33543612a36 None None] MFA Rules not processed for user `796772cf491047c8b0a609dae9402307`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:56.830601 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-298ca78f-62eb-47f3-aff3-a33543612a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:56.937512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 711/1411] 10.222.0.45 () {58 vars in 1015 bytes} [Thu Aug 4 02:10:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:56.952230 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355282f-d650-480c-84dc-9766cd45bb28 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:56.952514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355282f-d650-480c-84dc-9766cd45bb28 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:56.952825 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355282f-d650-480c-84dc-9766cd45bb28 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:56.954682 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 701/1412] 10.222.0.45 () {54 vars in 918 bytes} [Thu Aug 4 02:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:56.959192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bd30f-e1ec-4476-808a-332c470cbc71 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:56.959481 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bd30f-e1ec-4476-808a-332c470cbc71 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:56.959756 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bd30f-e1ec-4476-808a-332c470cbc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:57.030294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:57.034150 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-b81bd30f-e1ec-4476-808a-332c470cbc71 None None] MFA Rules not processed for user `c571406d2a3f40dab035a918399d6f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:57.035965 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b81bd30f-e1ec-4476-808a-332c470cbc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:57.160231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 712/1413] 10.222.0.45 () {58 vars in 1015 bytes} [Thu Aug 4 02:10:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:59.646859 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b2770-ce02-42fa-8df9-c88dcabeb1e3 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:59.647421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b2770-ce02-42fa-8df9-c88dcabeb1e3 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:59.647421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7b2770-ce02-42fa-8df9-c88dcabeb1e3 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:59.649231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 702/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 02:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:10:59.654539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e423d-6735-44d4-b140-a49f5df7b647 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:10:59.654968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e423d-6735-44d4-b140-a49f5df7b647 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:10:59.655097 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6e423d-6735-44d4-b140-a49f5df7b647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:10:59.721418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-1c6e423d-6735-44d4-b140-a49f5df7b647 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:10:59.722662 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1c6e423d-6735-44d4-b140-a49f5df7b647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:10:59.776753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:10:59.823312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 713/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 02:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:10:59.868663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b2770-ce02-42fa-8df9-c88dcabeb1e3 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:10:59.870184 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7c7b2770-ce02-42fa-8df9-c88dcabeb1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:00.013252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7b2770-ce02-42fa-8df9-c88dcabeb1e3 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:00.084677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:00.123612 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd67830-53c5-4db1-b5ca-d90f69c09c93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:00.124239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd67830-53c5-4db1-b5ca-d90f69c09c93 service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:00.124495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd67830-53c5-4db1-b5ca-d90f69c09c93 service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:00.124791 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd67830-53c5-4db1-b5ca-d90f69c09c93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:00.126923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9cd67830-53c5-4db1-b5ca-d90f69c09c93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:00.221161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd67830-53c5-4db1-b5ca-d90f69c09c93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:00.222768 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd67830-53c5-4db1-b5ca-d90f69c09c93 service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:00.240334 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 703/1416] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:10:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:01.148480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ffd06-c57f-4ce8-b900-f644f4f2a4b0 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:01.149053 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ffd06-c57f-4ce8-b900-f644f4f2a4b0 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:01.149179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68ffd06-c57f-4ce8-b900-f644f4f2a4b0 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:01.150491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 714/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 02:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:01.155451 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f10dde-6859-444a-82f5-870e530d63e1 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:01.155926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f10dde-6859-444a-82f5-870e530d63e1 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:01.156034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f10dde-6859-444a-82f5-870e530d63e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:01.231247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-86f10dde-6859-444a-82f5-870e530d63e1 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:01.232472 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-86f10dde-6859-444a-82f5-870e530d63e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:01.259608 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:01.329104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 704/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 02:11:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:01.480781 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ffd06-c57f-4ce8-b900-f644f4f2a4b0 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:01.482476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f68ffd06-c57f-4ce8-b900-f644f4f2a4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:01.628437 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68ffd06-c57f-4ce8-b900-f644f4f2a4b0 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:01.669818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:01.741646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c86db-6ca9-4303-a4fd-a9b95637a67d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:01.742171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c86db-6ca9-4303-a4fd-a9b95637a67d service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:01.742444 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c86db-6ca9-4303-a4fd-a9b95637a67d service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:01.742706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c86db-6ca9-4303-a4fd-a9b95637a67d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:01.744304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f60c86db-6ca9-4303-a4fd-a9b95637a67d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:01.832765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c86db-6ca9-4303-a4fd-a9b95637a67d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:01.834397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c86db-6ca9-4303-a4fd-a9b95637a67d service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:01.853037 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 715/1419] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:02.728671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a81f6-05a1-4ead-a89f-00f1f4b169b9 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:02.729215 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a81f6-05a1-4ead-a89f-00f1f4b169b9 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:02.729314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a81f6-05a1-4ead-a89f-00f1f4b169b9 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:02.730606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 705/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 02:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:02.736238 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b91017-76ef-4907-b350-cb103a1c396a None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:02.736668 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b91017-76ef-4907-b350-cb103a1c396a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:02.736868 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b91017-76ef-4907-b350-cb103a1c396a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:02.796727 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:02.819401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c4b91017-76ef-4907-b350-cb103a1c396a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:02.821000 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c4b91017-76ef-4907-b350-cb103a1c396a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:02.920377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 716/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 02:11:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:03.074361 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a81f6-05a1-4ead-a89f-00f1f4b169b9 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:03.075885 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-f31a81f6-05a1-4ead-a89f-00f1f4b169b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:03.221599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a81f6-05a1-4ead-a89f-00f1f4b169b9 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:03.233453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:03.334224 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fa700a-36ac-4c5a-bf60-74aadf90e5ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:03.334837 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa700a-36ac-4c5a-bf60-74aadf90e5ba service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:03.335111 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa700a-36ac-4c5a-bf60-74aadf90e5ba service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:03.335380 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fa700a-36ac-4c5a-bf60-74aadf90e5ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:03.337045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-53fa700a-36ac-4c5a-bf60-74aadf90e5ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:03.424817 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa700a-36ac-4c5a-bf60-74aadf90e5ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:03.426433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fa700a-36ac-4c5a-bf60-74aadf90e5ba service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:03.445020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 706/1422] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:04.318510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4b6c6-5188-4ea1-bde6-d256a95e1ea1 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:04.319074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4b6c6-5188-4ea1-bde6-d256a95e1ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:04.319074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4b6c6-5188-4ea1-bde6-d256a95e1ea1 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:04.320387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 717/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 02:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:11:04.325825 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ccf9f-943a-494c-a80e-26096f1e8329 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:04.326221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ccf9f-943a-494c-a80e-26096f1e8329 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:04.326372 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ccf9f-943a-494c-a80e-26096f1e8329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:04.346788 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:04.407934 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-124ccf9f-943a-494c-a80e-26096f1e8329 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:04.409170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-124ccf9f-943a-494c-a80e-26096f1e8329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:04.504296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 707/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 02:11:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:04.655514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4b6c6-5188-4ea1-bde6-d256a95e1ea1 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:04.657168 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f0e4b6c6-5188-4ea1-bde6-d256a95e1ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:04.776809 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:04.813726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4b6c6-5188-4ea1-bde6-d256a95e1ea1 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:04.928408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eea51f-d3f2-4215-b308-9257db2fddf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:04.929049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eea51f-d3f2-4215-b308-9257db2fddf7 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:04.929329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eea51f-d3f2-4215-b308-9257db2fddf7 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:04.929601 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eea51f-d3f2-4215-b308-9257db2fddf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:04.931368 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f8eea51f-d3f2-4215-b308-9257db2fddf7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:05.010971 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:05.024852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eea51f-d3f2-4215-b308-9257db2fddf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:05.026453 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eea51f-d3f2-4215-b308-9257db2fddf7 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:05.044293 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 718/1425] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:05.918623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504533f-346e-4ed5-b07c-87bb57d3e4b1 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:05.919140 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504533f-346e-4ed5-b07c-87bb57d3e4b1 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:05.919140 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504533f-346e-4ed5-b07c-87bb57d3e4b1 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:05.920481 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 708/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 02:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:05.925397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc5cc3-7095-4cda-bc34-f2ddc182945b None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:05.925870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc5cc3-7095-4cda-bc34-f2ddc182945b None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:05.925986 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc5cc3-7095-4cda-bc34-f2ddc182945b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:06.004397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-edcc5cc3-7095-4cda-bc34-f2ddc182945b None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:06.005657 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-edcc5cc3-7095-4cda-bc34-f2ddc182945b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:06.102550 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 719/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 02:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:06.253309 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504533f-346e-4ed5-b07c-87bb57d3e4b1 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:06.254885 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d504533f-346e-4ed5-b07c-87bb57d3e4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:06.339627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:06.400757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504533f-346e-4ed5-b07c-87bb57d3e4b1 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:06.510304 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88edd1-6f32-405c-bb52-a77643b697e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:06.510843 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88edd1-6f32-405c-bb52-a77643b697e1 service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:06.511129 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88edd1-6f32-405c-bb52-a77643b697e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:06.511384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88edd1-6f32-405c-bb52-a77643b697e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:06.513048 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-cb88edd1-6f32-405c-bb52-a77643b697e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:06.561243 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:06.604318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88edd1-6f32-405c-bb52-a77643b697e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:06.605948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88edd1-6f32-405c-bb52-a77643b697e1 service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:06.624152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 709/1428] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:07.428448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e8725-06f9-4cfa-91d9-7faef5332cf2 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:07.429032 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e8725-06f9-4cfa-91d9-7faef5332cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:07.429032 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5e8725-06f9-4cfa-91d9-7faef5332cf2 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:07.430441 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 720/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 02:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:07.435308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6b213-0c09-43c7-b3b6-ba2fd6035469 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:07.435765 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6b213-0c09-43c7-b3b6-ba2fd6035469 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:07.435873 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6b213-0c09-43c7-b3b6-ba2fd6035469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:07.519206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-fbf6b213-0c09-43c7-b3b6-ba2fd6035469 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:07.520565 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-fbf6b213-0c09-43c7-b3b6-ba2fd6035469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:07.618671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 710/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 02:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:07.769109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e8725-06f9-4cfa-91d9-7faef5332cf2 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:07.770545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-dc5e8725-06f9-4cfa-91d9-7faef5332cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:07.818167 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:07.920702 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5e8725-06f9-4cfa-91d9-7faef5332cf2 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:08.035454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a2c64-8d08-4e41-be5b-c15d301d3218 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:08.035976 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a2c64-8d08-4e41-be5b-c15d301d3218 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:08.036254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a2c64-8d08-4e41-be5b-c15d301d3218 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:08.036529 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a2c64-8d08-4e41-be5b-c15d301d3218 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:08.038200 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b24a2c64-8d08-4e41-be5b-c15d301d3218 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:08.045106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:08.129387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a2c64-8d08-4e41-be5b-c15d301d3218 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:08.130974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a2c64-8d08-4e41-be5b-c15d301d3218 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:08.148675 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 721/1431] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:08.967873 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e58106-6af8-49ac-8110-253bfca739f5 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:08.968442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e58106-6af8-49ac-8110-253bfca739f5 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:08.968442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e58106-6af8-49ac-8110-253bfca739f5 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:08.970412 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 711/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 02:11:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:08.975426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a1a82-6f63-4ed9-a8e4-3ceac7918595 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:08.975832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a1a82-6f63-4ed9-a8e4-3ceac7918595 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:08.975977 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a1a82-6f63-4ed9-a8e4-3ceac7918595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:09.050851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-d13a1a82-6f63-4ed9-a8e4-3ceac7918595 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:09.052055 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d13a1a82-6f63-4ed9-a8e4-3ceac7918595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:09.119963 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:09.150159 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 722/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 02:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:09.300387 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e58106-6af8-49ac-8110-253bfca739f5 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:09.302202 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-37e58106-6af8-49ac-8110-253bfca739f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:09.325006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:09.456554 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e58106-6af8-49ac-8110-253bfca739f5 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:09.541420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:09.575181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b159d-9480-4c3e-9c17-65fddb1aa723 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:09.576148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b159d-9480-4c3e-9c17-65fddb1aa723 service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:09.576702 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b159d-9480-4c3e-9c17-65fddb1aa723 service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:09.577224 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b159d-9480-4c3e-9c17-65fddb1aa723 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:09.579419 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-086b159d-9480-4c3e-9c17-65fddb1aa723 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:09.681322 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b159d-9480-4c3e-9c17-65fddb1aa723 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:09.682932 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b159d-9480-4c3e-9c17-65fddb1aa723 service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:09.701440 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 712/1434] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:10.519954 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5236a-63c9-4a3f-b4f3-09d17c1af13a None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:10.520511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5236a-63c9-4a3f-b4f3-09d17c1af13a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:10.520511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5236a-63c9-4a3f-b4f3-09d17c1af13a None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:10.521832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 723/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 02:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:10.526843 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508525f0-8f37-4d91-8ef8-f968ff73308c None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:10.527282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508525f0-8f37-4d91-8ef8-f968ff73308c None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:10.527396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508525f0-8f37-4d91-8ef8-f968ff73308c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:10.602731 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-508525f0-8f37-4d91-8ef8-f968ff73308c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:10.603956 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-508525f0-8f37-4d91-8ef8-f968ff73308c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:10.644966 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:10.699238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 713/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 02:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:10.851424 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5236a-63c9-4a3f-b4f3-09d17c1af13a None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:10.852918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-69c5236a-63c9-4a3f-b4f3-09d17c1af13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:11.117793 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5236a-63c9-4a3f-b4f3-09d17c1af13a None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:11.173393 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:11.232457 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e1105e-cea1-4552-be4d-4abf5c294cb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:11.233101 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1105e-cea1-4552-be4d-4abf5c294cb1 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:11.233379 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1105e-cea1-4552-be4d-4abf5c294cb1 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:11.233649 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e1105e-cea1-4552-be4d-4abf5c294cb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:11.235767 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f5e1105e-cea1-4552-be4d-4abf5c294cb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:11.328347 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e1105e-cea1-4552-be4d-4abf5c294cb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:11.330030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e1105e-cea1-4552-be4d-4abf5c294cb1 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:11.349277 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 724/1437] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:12.148267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47368af3-ff28-4c6a-8c38-832cb6ca305d None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:12.148837 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47368af3-ff28-4c6a-8c38-832cb6ca305d None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:12.148837 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47368af3-ff28-4c6a-8c38-832cb6ca305d None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:12.150238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 714/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 02:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:12.155322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f013d4c-2080-4fd0-a119-d55a966e294e None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:12.155755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f013d4c-2080-4fd0-a119-d55a966e294e None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:12.155876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f013d4c-2080-4fd0-a119-d55a966e294e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:12.235193 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-2f013d4c-2080-4fd0-a119-d55a966e294e None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:12.236430 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-2f013d4c-2080-4fd0-a119-d55a966e294e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:12.238572 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:12.338903 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 725/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 02:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:12.488166 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47368af3-ff28-4c6a-8c38-832cb6ca305d None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:12.489645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-47368af3-ff28-4c6a-8c38-832cb6ca305d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:12.637232 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47368af3-ff28-4c6a-8c38-832cb6ca305d None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:12.664762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:12.749942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ff4c6a-4a57-4062-8d73-2c8eb7c9aba0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:12.751136 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff4c6a-4a57-4062-8d73-2c8eb7c9aba0 service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:12.751410 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff4c6a-4a57-4062-8d73-2c8eb7c9aba0 service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:12.751676 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ff4c6a-4a57-4062-8d73-2c8eb7c9aba0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:12.753333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-82ff4c6a-4a57-4062-8d73-2c8eb7c9aba0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:12.842082 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff4c6a-4a57-4062-8d73-2c8eb7c9aba0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:12.843679 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ff4c6a-4a57-4062-8d73-2c8eb7c9aba0 service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:12.868423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 715/1440] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:14.816326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17db0a0-c1e3-4163-abba-0bdc611a5ed7 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:14.816908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17db0a0-c1e3-4163-abba-0bdc611a5ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:14.816908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17db0a0-c1e3-4163-abba-0bdc611a5ed7 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:14.818294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 726/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 02:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:14.823917 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5af61d-2e4e-425c-a404-2533658afe44 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:14.824331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5af61d-2e4e-425c-a404-2533658afe44 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:14.824487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5af61d-2e4e-425c-a404-2533658afe44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:14.865793 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:14.892257 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-6d5af61d-2e4e-425c-a404-2533658afe44 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:14.893526 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-6d5af61d-2e4e-425c-a404-2533658afe44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:14.990432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 716/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 02:11:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:15.028983 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17db0a0-c1e3-4163-abba-0bdc611a5ed7 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:15.030513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a17db0a0-c1e3-4163-abba-0bdc611a5ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:15.133657 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17db0a0-c1e3-4163-abba-0bdc611a5ed7 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:15.165219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:15.206684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb439a6-a91a-4e87-a9f6-45b169702ca5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:15.207297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb439a6-a91a-4e87-a9f6-45b169702ca5 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:15.207574 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb439a6-a91a-4e87-a9f6-45b169702ca5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:15.207841 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb439a6-a91a-4e87-a9f6-45b169702ca5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:15.209490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-5bb439a6-a91a-4e87-a9f6-45b169702ca5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:15.323018 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb439a6-a91a-4e87-a9f6-45b169702ca5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:15.324352 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb439a6-a91a-4e87-a9f6-45b169702ca5 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:15.342653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 727/1443] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:17.530305 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eea03d-2591-4128-91f8-de0bee9b55c7 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:17.530956 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eea03d-2591-4128-91f8-de0bee9b55c7 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:17.530956 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eea03d-2591-4128-91f8-de0bee9b55c7 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:17.532763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 717/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 02:11:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:17.537988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe6649-ead6-40ff-b27d-ffe2070c2ec3 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:17.538446 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe6649-ead6-40ff-b27d-ffe2070c2ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:17.538569 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe6649-ead6-40ff-b27d-ffe2070c2ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:17.604326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-eabe6649-ead6-40ff-b27d-ffe2070c2ec3 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:17.605561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-eabe6649-ead6-40ff-b27d-ffe2070c2ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:17.619085 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:17.704676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 728/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 02:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:17.743642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eea03d-2591-4128-91f8-de0bee9b55c7 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:17.745234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-46eea03d-2591-4128-91f8-de0bee9b55c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:17.846709 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eea03d-2591-4128-91f8-de0bee9b55c7 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:17.858701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:17.920273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c11c166-7f0e-4e9f-8050-c0678b5254dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:17.920918 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11c166-7f0e-4e9f-8050-c0678b5254dd service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:17.921201 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11c166-7f0e-4e9f-8050-c0678b5254dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:17.921476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c11c166-7f0e-4e9f-8050-c0678b5254dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:17.923169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-0c11c166-7f0e-4e9f-8050-c0678b5254dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:18.034816 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11c166-7f0e-4e9f-8050-c0678b5254dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:18.036256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c11c166-7f0e-4e9f-8050-c0678b5254dd service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:18.054154 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 718/1446] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:20.469818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812573f-31f4-421c-80ab-91ded1e1ccf3 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:20.470348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812573f-31f4-421c-80ab-91ded1e1ccf3 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:20.470348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812573f-31f4-421c-80ab-91ded1e1ccf3 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:20.471645 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 729/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 02:11:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:20.477049 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff715671-393b-4cf6-a35c-fe53ca03c119 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:20.477448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff715671-393b-4cf6-a35c-fe53ca03c119 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:20.477600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff715671-393b-4cf6-a35c-fe53ca03c119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:20.526210 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:20.546330 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-ff715671-393b-4cf6-a35c-fe53ca03c119 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:20.548182 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ff715671-393b-4cf6-a35c-fe53ca03c119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:20.643637 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 719/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 02:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:20.683376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812573f-31f4-421c-80ab-91ded1e1ccf3 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:20.684994 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f812573f-31f4-421c-80ab-91ded1e1ccf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:20.787064 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812573f-31f4-421c-80ab-91ded1e1ccf3 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:20.829136 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:20.863662 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d951a-53e3-45cf-a622-d9d9aa68d7ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:20.864272 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d951a-53e3-45cf-a622-d9d9aa68d7ee service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:20.864550 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d951a-53e3-45cf-a622-d9d9aa68d7ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:20.864851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d951a-53e3-45cf-a622-d9d9aa68d7ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:20.866476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8f5d951a-53e3-45cf-a622-d9d9aa68d7ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:20.974492 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d951a-53e3-45cf-a622-d9d9aa68d7ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:20.975926 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d951a-53e3-45cf-a622-d9d9aa68d7ee service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:20.993642 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 730/1449] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:23.123414 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3378d4-98ee-496a-955c-edb60c59702b None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:23.123929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3378d4-98ee-496a-955c-edb60c59702b None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:23.124071 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3378d4-98ee-496a-955c-edb60c59702b None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:23.126008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 720/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 02:11:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:23.131361 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda4a7b-2f23-46d1-8fbd-644510475536 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:23.131768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda4a7b-2f23-46d1-8fbd-644510475536 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:23.131909 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdda4a7b-2f23-46d1-8fbd-644510475536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:23.198135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-bdda4a7b-2f23-46d1-8fbd-644510475536 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:23.199349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-bdda4a7b-2f23-46d1-8fbd-644510475536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:23.227196 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:23.303039 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 731/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 02:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:23.343449 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3378d4-98ee-496a-955c-edb60c59702b None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:23.345023 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9c3378d4-98ee-496a-955c-edb60c59702b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:23.453629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3378d4-98ee-496a-955c-edb60c59702b None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:23.475466 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:23.528895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523dd29-d123-47b1-8360-aba3748de681 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:23.529521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523dd29-d123-47b1-8360-aba3748de681 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:23.529803 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523dd29-d123-47b1-8360-aba3748de681 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:23.530084 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523dd29-d123-47b1-8360-aba3748de681 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:23.532214 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2523dd29-d123-47b1-8360-aba3748de681 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:23.639831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523dd29-d123-47b1-8360-aba3748de681 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:23.641324 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523dd29-d123-47b1-8360-aba3748de681 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:23.660273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 721/1452] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:25.761615 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7756c0-f6b2-4dd5-8b62-b0d0cb1a885a None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:25.762176 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7756c0-f6b2-4dd5-8b62-b0d0cb1a885a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:25.762176 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7756c0-f6b2-4dd5-8b62-b0d0cb1a885a None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:25.763584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 732/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 02:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:25.769530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85809e1-fc9d-4d8b-9505-e291e6853542 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:25.769923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85809e1-fc9d-4d8b-9505-e291e6853542 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:25.770086 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85809e1-fc9d-4d8b-9505-e291e6853542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:25.837319 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:25.840610 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-e85809e1-fc9d-4d8b-9505-e291e6853542 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:25.842443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e85809e1-fc9d-4d8b-9505-e291e6853542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:26.067002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 722/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 02:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:26.105539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7756c0-f6b2-4dd5-8b62-b0d0cb1a885a None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:26.107074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-3c7756c0-f6b2-4dd5-8b62-b0d0cb1a885a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:26.209467 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7756c0-f6b2-4dd5-8b62-b0d0cb1a885a None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:26.258283 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:26.283100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea2cd23-cd86-4793-b736-964f66dbe2c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:26.283711 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2cd23-cd86-4793-b736-964f66dbe2c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:26.283988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2cd23-cd86-4793-b736-964f66dbe2c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:26.284287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea2cd23-cd86-4793-b736-964f66dbe2c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:26.285950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-0ea2cd23-cd86-4793-b736-964f66dbe2c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:26.394517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea2cd23-cd86-4793-b736-964f66dbe2c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:26.395943 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea2cd23-cd86-4793-b736-964f66dbe2c8 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:26.414400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 733/1455] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:28.500000 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b49f3-ebe4-4eb6-ae71-929eb6e1f00b None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:28.501087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b49f3-ebe4-4eb6-ae71-929eb6e1f00b None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:28.501087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b49f3-ebe4-4eb6-ae71-929eb6e1f00b None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:28.502426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 723/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 02:11:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:28.507947 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98727d-4306-4cf9-8337-1c8d9833daa0 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:28.508401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98727d-4306-4cf9-8337-1c8d9833daa0 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:28.508500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98727d-4306-4cf9-8337-1c8d9833daa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:28.577914 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-4c98727d-4306-4cf9-8337-1c8d9833daa0 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:28.579156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4c98727d-4306-4cf9-8337-1c8d9833daa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:28.615805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:28.685131 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 734/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 02:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:28.723397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b49f3-ebe4-4eb6-ae71-929eb6e1f00b None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:28.724978 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-794b49f3-ebe4-4eb6-ae71-929eb6e1f00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:28.825634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b49f3-ebe4-4eb6-ae71-929eb6e1f00b None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:28.853184 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:28.898866 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240b90b-0512-4db0-8d48-ccca5e5a86fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:28.899506 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240b90b-0512-4db0-8d48-ccca5e5a86fc service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:28.899787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240b90b-0512-4db0-8d48-ccca5e5a86fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:28.900062 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240b90b-0512-4db0-8d48-ccca5e5a86fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:28.901791 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e240b90b-0512-4db0-8d48-ccca5e5a86fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:29.012748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240b90b-0512-4db0-8d48-ccca5e5a86fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:29.014185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240b90b-0512-4db0-8d48-ccca5e5a86fc service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:29.033118 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 724/1458] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:31.103031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0fc89-38de-4c9d-aee1-d2fb45ae36bf None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:31.103582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0fc89-38de-4c9d-aee1-d2fb45ae36bf None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:31.103582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0fc89-38de-4c9d-aee1-d2fb45ae36bf None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:31.104969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 735/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 02:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:31.110166 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a23f5-052c-4989-a829-bf6b334636c6 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:31.110594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a23f5-052c-4989-a829-bf6b334636c6 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:31.110717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a23f5-052c-4989-a829-bf6b334636c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:31.178953 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-9a0a23f5-052c-4989-a829-bf6b334636c6 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:31.180171 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9a0a23f5-052c-4989-a829-bf6b334636c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:31.188807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:31.277884 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 725/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 02:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:31.317016 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0fc89-38de-4c9d-aee1-d2fb45ae36bf None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:31.318655 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-97b0fc89-38de-4c9d-aee1-d2fb45ae36bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:31.421090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b0fc89-38de-4c9d-aee1-d2fb45ae36bf None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:31.475893 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:31.494783 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c4655a-fe53-4a09-b140-9cf8972869bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:31.495393 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4655a-fe53-4a09-b140-9cf8972869bb service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:31.495698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4655a-fe53-4a09-b140-9cf8972869bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:31.495944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c4655a-fe53-4a09-b140-9cf8972869bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:31.497592 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a3c4655a-fe53-4a09-b140-9cf8972869bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:31.606382 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4655a-fe53-4a09-b140-9cf8972869bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:31.607818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c4655a-fe53-4a09-b140-9cf8972869bb service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:31.626776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 736/1461] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:33.695002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b90af7-8d43-4e0b-85ad-ed91c73d2fd8 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:33.695518 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b90af7-8d43-4e0b-85ad-ed91c73d2fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:33.695635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b90af7-8d43-4e0b-85ad-ed91c73d2fd8 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:33.697230 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 726/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 02:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:33.703801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01396b-7f81-425a-9125-c0e36435244e None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:33.704227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01396b-7f81-425a-9125-c0e36435244e None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:33.704341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b01396b-7f81-425a-9125-c0e36435244e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:33.770514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-4b01396b-7f81-425a-9125-c0e36435244e None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:33.771735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4b01396b-7f81-425a-9125-c0e36435244e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:33.809992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:33.868451 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 737/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 02:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:33.906814 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b90af7-8d43-4e0b-85ad-ed91c73d2fd8 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:33.908409 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a0b90af7-8d43-4e0b-85ad-ed91c73d2fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:34.009088 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b90af7-8d43-4e0b-85ad-ed91c73d2fd8 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:34.045254 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:34.080939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be1ae9-1511-44c5-8063-bcdb4e455723 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:34.081549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be1ae9-1511-44c5-8063-bcdb4e455723 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:34.081823 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be1ae9-1511-44c5-8063-bcdb4e455723 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:34.082098 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be1ae9-1511-44c5-8063-bcdb4e455723 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:34.083729 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-62be1ae9-1511-44c5-8063-bcdb4e455723 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:34.189477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be1ae9-1511-44c5-8063-bcdb4e455723 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:34.190896 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be1ae9-1511-44c5-8063-bcdb4e455723 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:34.210266 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 727/1464] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:36.237023 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13a72d-3db2-40d3-a958-497395b9ec8c None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:36.237527 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13a72d-3db2-40d3-a958-497395b9ec8c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:36.237619 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13a72d-3db2-40d3-a958-497395b9ec8c None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:36.238935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 738/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 02:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:36.244253 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc622c9-7b7e-4490-99d9-5e2bd2a8b9c7 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:36.244702 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc622c9-7b7e-4490-99d9-5e2bd2a8b9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:36.244826 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc622c9-7b7e-4490-99d9-5e2bd2a8b9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:36.310223 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-2bc622c9-7b7e-4490-99d9-5e2bd2a8b9c7 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:36.311435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2bc622c9-7b7e-4490-99d9-5e2bd2a8b9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:36.330048 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:36.408222 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 728/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 02:11:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:36.445927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13a72d-3db2-40d3-a958-497395b9ec8c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:36.447179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-cf13a72d-3db2-40d3-a958-497395b9ec8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:36.549365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13a72d-3db2-40d3-a958-497395b9ec8c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:36.614654 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:36.622389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63602ff-c45e-496b-af5b-3b833a8881c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:36.623007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63602ff-c45e-496b-af5b-3b833a8881c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:36.623289 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63602ff-c45e-496b-af5b-3b833a8881c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:36.623562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63602ff-c45e-496b-af5b-3b833a8881c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:36.625279 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c63602ff-c45e-496b-af5b-3b833a8881c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:36.733739 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63602ff-c45e-496b-af5b-3b833a8881c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:36.735093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63602ff-c45e-496b-af5b-3b833a8881c3 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:36.753006 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 739/1467] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:38.824177 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848b0f8-d253-4926-b70c-4a86be59fb42 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:38.824729 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848b0f8-d253-4926-b70c-4a86be59fb42 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:38.824847 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8848b0f8-d253-4926-b70c-4a86be59fb42 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:38.826110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 729/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 02:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:38.831285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1a30e-31f9-4e10-89ce-ece0078f7a49 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:38.831751 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1a30e-31f9-4e10-89ce-ece0078f7a49 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:38.831860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1a30e-31f9-4e10-89ce-ece0078f7a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:38.897213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-0ab1a30e-31f9-4e10-89ce-ece0078f7a49 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:38.898436 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-0ab1a30e-31f9-4e10-89ce-ece0078f7a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:38.946984 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:38.995505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 740/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 02:11:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:11:39.032155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848b0f8-d253-4926-b70c-4a86be59fb42 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:11:39.033781 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-8848b0f8-d253-4926-b70c-4a86be59fb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:39.136813 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8848b0f8-d253-4926-b70c-4a86be59fb42 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:11:39.183523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:39.209227 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2f884-3107-480d-8bf2-3fbd3fb2a212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:11:39.209770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2f884-3107-480d-8bf2-3fbd3fb2a212 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:39.210049 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2f884-3107-480d-8bf2-3fbd3fb2a212 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:39.210319 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2f884-3107-480d-8bf2-3fbd3fb2a212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:39.211934 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7ab2f884-3107-480d-8bf2-3fbd3fb2a212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:39.321243 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2f884-3107-480d-8bf2-3fbd3fb2a212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:11:39.323083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2f884-3107-480d-8bf2-3fbd3fb2a212 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:11:39.343959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 730/1470] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:11:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:11:48.993762 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=22 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:49.340950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=66 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:11:59.801842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8ca73-9a46-438e-8115-222cc8ddbed4 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:59.802394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8ca73-9a46-438e-8115-222cc8ddbed4 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:59.802394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8ca73-9a46-438e-8115-222cc8ddbed4 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:59.803755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 741/1471] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:11:59.808931 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e411e4f-44f6-429c-8696-18c2dc14a799 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:11:59.809380 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e411e4f-44f6-429c-8696-18c2dc14a799 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:11:59.809477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e411e4f-44f6-429c-8696-18c2dc14a799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:11:59.874056 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-9e411e4f-44f6-429c-8696-18c2dc14a799 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:11:59.875240 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9e411e4f-44f6-429c-8696-18c2dc14a799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:11:59.975575 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 731/1472] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:12:00.058956 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6593c-d23d-4854-a26e-948fb0c2eccd None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:00.059241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6593c-d23d-4854-a26e-948fb0c2eccd None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:00.059522 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6593c-d23d-4854-a26e-948fb0c2eccd None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:00.060933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 742/1473] 10.222.0.45 () {54 vars in 966 bytes} [Thu Aug 4 02:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:12:00.065864 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a79194-59e1-456b-bfd5-b318e6d4f753 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:00.066149 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a79194-59e1-456b-bfd5-b318e6d4f753 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:00.066423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a79194-59e1-456b-bfd5-b318e6d4f753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:00.140835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-95a79194-59e1-456b-bfd5-b318e6d4f753 None None] MFA Rules not processed for user `14ff31d52ecf4c7d93a65bbfca7285c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:12:00.142627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-95a79194-59e1-456b-bfd5-b318e6d4f753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:00.168118 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:12:00.225151 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 732/1474] 10.222.0.45 () {58 vars in 1063 bytes} [Thu Aug 4 02:12:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:12:00.232654 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6593c-d23d-4854-a26e-948fb0c2eccd None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:12:00.234123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-15b6593c-d23d-4854-a26e-948fb0c2eccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:00.338243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6593c-d23d-4854-a26e-948fb0c2eccd None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:12:00.410187 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb6d8a6-c57e-40fc-8f09-8b0cf3050f66 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ff31d52ecf4c7d93a65bbfca7285c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:12:00.410797 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6d8a6-c57e-40fc-8f09-8b0cf3050f66 service glance] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:00.411076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6d8a6-c57e-40fc-8f09-8b0cf3050f66 service glance] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:00.411349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb6d8a6-c57e-40fc-8f09-8b0cf3050f66 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:00.412994 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-eeb6d8a6-c57e-40fc-8f09-8b0cf3050f66 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:00.467671 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:12:00.522903 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6d8a6-c57e-40fc-8f09-8b0cf3050f66 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:12:00.524238 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb6d8a6-c57e-40fc-8f09-8b0cf3050f66 service glance] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:12:00.542570 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 743/1475] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:12:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:12:02.643300 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9455c-62f0-41a9-9ed4-c025d8c1fa09 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:02.643877 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9455c-62f0-41a9-9ed4-c025d8c1fa09 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:02.643877 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9455c-62f0-41a9-9ed4-c025d8c1fa09 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:02.645648 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 733/1476] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:12:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:12:02.651366 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383679e6-479d-490a-a376-77abc683fb75 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:02.651829 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383679e6-479d-490a-a376-77abc683fb75 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:02.651941 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383679e6-479d-490a-a376-77abc683fb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:02.717692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-383679e6-479d-490a-a376-77abc683fb75 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:12:02.718900 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-383679e6-479d-490a-a376-77abc683fb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:02.820327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 744/1477] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:12:02.845005 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9455c-62f0-41a9-9ed4-c025d8c1fa09 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:12:02.846541 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-68c9455c-62f0-41a9-9ed4-c025d8c1fa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:02.971145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9455c-62f0-41a9-9ed4-c025d8c1fa09 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:12:03.019504 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:12:03.062822 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3115d9-e819-45ef-ab7a-fb0a8541f107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:12:03.064060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3115d9-e819-45ef-ab7a-fb0a8541f107 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:03.064333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3115d9-e819-45ef-ab7a-fb0a8541f107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:03.064606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3115d9-e819-45ef-ab7a-fb0a8541f107 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:03.065317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3115d9-e819-45ef-ab7a-fb0a8541f107 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:12:03.067042 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: WARNING py.warnings [None req-7f3115d9-e819-45ef-ab7a-fb0a8541f107 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 02:12:03.067042 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: warnings.warn(msg) Aug 04 02:12:03.067042 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: Aug 04 02:12:03.068174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3115d9-e819-45ef-ab7a-fb0a8541f107 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:12:03.086588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 734/1478] 10.222.0.45 () {56 vars in 1108 bytes} [Thu Aug 4 02:12:02 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:12:05.108895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e2276-4456-44e6-b725-8edcf9847b53 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:05.109419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e2276-4456-44e6-b725-8edcf9847b53 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:05.109419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35e2276-4456-44e6-b725-8edcf9847b53 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:05.110767 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 745/1479] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:12:05.116012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a34bc-41b2-4ec6-8a17-4bb8b6cc156a None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:05.116415 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a34bc-41b2-4ec6-8a17-4bb8b6cc156a None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:05.116566 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a34bc-41b2-4ec6-8a17-4bb8b6cc156a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:05.181360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-f77a34bc-41b2-4ec6-8a17-4bb8b6cc156a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:12:05.182569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-f77a34bc-41b2-4ec6-8a17-4bb8b6cc156a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:05.277498 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 735/1480] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:12:05.321130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e2276-4456-44e6-b725-8edcf9847b53 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:12:05.322894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c35e2276-4456-44e6-b725-8edcf9847b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:05.332570 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:12:05.479048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35e2276-4456-44e6-b725-8edcf9847b53 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:12:05.569389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:12:05.604552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dc68e1-425d-4126-b0b4-0973495cdcdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:12:05.605194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc68e1-425d-4126-b0b4-0973495cdcdd service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:05.605477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc68e1-425d-4126-b0b4-0973495cdcdd service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:05.605755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dc68e1-425d-4126-b0b4-0973495cdcdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:05.607403 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-83dc68e1-425d-4126-b0b4-0973495cdcdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:05.701417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc68e1-425d-4126-b0b4-0973495cdcdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:12:05.702995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dc68e1-425d-4126-b0b4-0973495cdcdd service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:12:05.721133 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 746/1481] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:12:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:12:07.908715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e399fa-3a93-43ac-b641-c7de9aa3ead5 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:07.909274 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e399fa-3a93-43ac-b641-c7de9aa3ead5 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:07.909274 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e399fa-3a93-43ac-b641-c7de9aa3ead5 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:07.910578 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 736/1482] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:12:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:12:07.915774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f550a5-1e4b-4141-9c28-64eab1d99695 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:07.916243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f550a5-1e4b-4141-9c28-64eab1d99695 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:07.916351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f550a5-1e4b-4141-9c28-64eab1d99695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:07.983199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-e7f550a5-1e4b-4141-9c28-64eab1d99695 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:12:07.984412 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e7f550a5-1e4b-4141-9c28-64eab1d99695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:08.030789 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:12:08.082651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 747/1483] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:12:08.119570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e399fa-3a93-43ac-b641-c7de9aa3ead5 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:12:08.121146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-01e399fa-3a93-43ac-b641-c7de9aa3ead5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:08.144796 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:12:08.237364 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e399fa-3a93-43ac-b641-c7de9aa3ead5 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:12:08.313523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958232dc-7e2b-429a-8a3c-2bde847fee50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:12:08.314145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958232dc-7e2b-429a-8a3c-2bde847fee50 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:08.314430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958232dc-7e2b-429a-8a3c-2bde847fee50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:08.314692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958232dc-7e2b-429a-8a3c-2bde847fee50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:08.316334 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-958232dc-7e2b-429a-8a3c-2bde847fee50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:08.390586 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:12:08.424732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958232dc-7e2b-429a-8a3c-2bde847fee50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:12:08.426165 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958232dc-7e2b-429a-8a3c-2bde847fee50 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:12:08.444010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 737/1484] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:12:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:12:10.515193 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5132616-1542-48c3-bf85-834240d188c8 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:10.515733 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5132616-1542-48c3-bf85-834240d188c8 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:10.515733 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5132616-1542-48c3-bf85-834240d188c8 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:10.517552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 748/1485] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:12:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:12:10.522875 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a0cec-ace7-4c92-ba0d-adad28c28196 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:10.523319 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a0cec-ace7-4c92-ba0d-adad28c28196 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:10.523445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a0cec-ace7-4c92-ba0d-adad28c28196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:10.593575 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-fa3a0cec-ace7-4c92-ba0d-adad28c28196 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:12:10.596113 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-fa3a0cec-ace7-4c92-ba0d-adad28c28196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:10.700514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 738/1486] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:12:10.739072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5132616-1542-48c3-bf85-834240d188c8 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:12:10.740596 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f5132616-1542-48c3-bf85-834240d188c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:10.847447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5132616-1542-48c3-bf85-834240d188c8 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:12:10.919807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:12:10.921680 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da5dc1-0528-43ae-ae52-4735998e99cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:12:10.922357 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da5dc1-0528-43ae-ae52-4735998e99cd service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:10.922658 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da5dc1-0528-43ae-ae52-4735998e99cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:10.923222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da5dc1-0528-43ae-ae52-4735998e99cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:10.925249 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e8da5dc1-0528-43ae-ae52-4735998e99cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:11.040599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da5dc1-0528-43ae-ae52-4735998e99cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:12:11.042039 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da5dc1-0528-43ae-ae52-4735998e99cd service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:12:11.060146 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 749/1487] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:12:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:12:13.238202 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88739295-5a7a-4253-bf25-bf200a97cc9a None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:13.238766 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88739295-5a7a-4253-bf25-bf200a97cc9a None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:13.238766 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88739295-5a7a-4253-bf25-bf200a97cc9a None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:13.240093 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 739/1488] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:12:13.245237 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba454204-9b75-4a4a-ba6e-159b2d592d49 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:13.245688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba454204-9b75-4a4a-ba6e-159b2d592d49 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:13.245784 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba454204-9b75-4a4a-ba6e-159b2d592d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:13.312102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-ba454204-9b75-4a4a-ba6e-159b2d592d49 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:12:13.313996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-ba454204-9b75-4a4a-ba6e-159b2d592d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:13.369510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:12:13.414271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 750/1489] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:12:13.454787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88739295-5a7a-4253-bf25-bf200a97cc9a None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:12:13.456386 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-88739295-5a7a-4253-bf25-bf200a97cc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:13.492781 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:12:13.563377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88739295-5a7a-4253-bf25-bf200a97cc9a None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:12:13.636499 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0400c-2f40-4faa-bbac-7e2ef428ec57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:12:13.637146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0400c-2f40-4faa-bbac-7e2ef428ec57 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:13.637429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0400c-2f40-4faa-bbac-7e2ef428ec57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:13.637700 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0400c-2f40-4faa-bbac-7e2ef428ec57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:13.639330 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-44d0400c-2f40-4faa-bbac-7e2ef428ec57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:13.724999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:12:13.754354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0400c-2f40-4faa-bbac-7e2ef428ec57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:12:13.755721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0400c-2f40-4faa-bbac-7e2ef428ec57 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:12:13.773528 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 740/1490] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:12:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:12:15.977160 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004afbe-ef4a-4842-b326-237415a792a9 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:15.977709 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004afbe-ef4a-4842-b326-237415a792a9 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:15.977709 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5004afbe-ef4a-4842-b326-237415a792a9 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:15.978964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 751/1491] 10.222.0.45 () {54 vars in 927 bytes} [Thu Aug 4 02:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:12:15.984139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ede74-a791-4034-a950-6db48c3892bf None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:15.984545 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ede74-a791-4034-a950-6db48c3892bf None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:15.984710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ede74-a791-4034-a950-6db48c3892bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:16.050513 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-eb8ede74-a791-4034-a950-6db48c3892bf None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:12:16.051719 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-eb8ede74-a791-4034-a950-6db48c3892bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:16.146732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 741/1492] 10.222.0.45 () {58 vars in 1024 bytes} [Thu Aug 4 02:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:12:16.185749 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004afbe-ef4a-4842-b326-237415a792a9 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:12:16.187281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-5004afbe-ef4a-4842-b326-237415a792a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:16.291734 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5004afbe-ef4a-4842-b326-237415a792a9 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:12:16.365550 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beb56dc-b259-4795-8124-5db9f679df7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:12:16.366164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb56dc-b259-4795-8124-5db9f679df7e service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:12:16.366446 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb56dc-b259-4795-8124-5db9f679df7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:12:16.366716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beb56dc-b259-4795-8124-5db9f679df7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:12:16.368372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-3beb56dc-b259-4795-8124-5db9f679df7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:12:16.378119 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:12:16.479072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb56dc-b259-4795-8124-5db9f679df7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:12:16.480514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beb56dc-b259-4795-8124-5db9f679df7e service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:12:16.499222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 752/1493] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:12:26.144863 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=82 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:12:26.496928 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=50 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:13:27.377729 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13cd65-241b-42c4-b699-5b0b2e23c025 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:13:27.378331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13cd65-241b-42c4-b699-5b0b2e23c025 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:13:27.378331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13cd65-241b-42c4-b699-5b0b2e23c025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:13:27.453600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-fa13cd65-241b-42c4-b699-5b0b2e23c025 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:13:27.454795 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-fa13cd65-241b-42c4-b699-5b0b2e23c025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:13:27.549681 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 742/1494] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:13:27.557891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af3e34-1fec-4b2a-8ee1-efcb6a88dc31 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:13:27.558382 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af3e34-1fec-4b2a-8ee1-efcb6a88dc31 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:13:27.558518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85af3e34-1fec-4b2a-8ee1-efcb6a88dc31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:13:27.633414 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-85af3e34-1fec-4b2a-8ee1-efcb6a88dc31 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:13:27.634556 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-85af3e34-1fec-4b2a-8ee1-efcb6a88dc31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:13:27.729823 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 753/1495] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:13:27.736218 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13cd65-241b-42c4-b699-5b0b2e23c025 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:13:27.737664 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-fa13cd65-241b-42c4-b699-5b0b2e23c025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:13:27.750527 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13cd65-241b-42c4-b699-5b0b2e23c025 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:13:27.812519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:13:27.846381 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f64641-c67a-42ab-8a7b-5dbbdac7ed81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:13:27.846995 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f64641-c67a-42ab-8a7b-5dbbdac7ed81 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:13:27.847275 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f64641-c67a-42ab-8a7b-5dbbdac7ed81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:13:27.847548 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f64641-c67a-42ab-8a7b-5dbbdac7ed81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:13:27.848238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f64641-c67a-42ab-8a7b-5dbbdac7ed81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:13:27.848541 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f64641-c67a-42ab-8a7b-5dbbdac7ed81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:13:27.849589 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: WARNING py.warnings [None req-05f64641-c67a-42ab-8a7b-5dbbdac7ed81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 02:13:27.849589 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: warnings.warn(msg) Aug 04 02:13:27.849589 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: Aug 04 02:13:27.850477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f64641-c67a-42ab-8a7b-5dbbdac7ed81 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:13:27.862051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 743/1496] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:13:27.870671 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af3e34-1fec-4b2a-8ee1-efcb6a88dc31 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:13:27.880696 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85af3e34-1fec-4b2a-8ee1-efcb6a88dc31 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:13:27.943829 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:13:27.978987 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559361f-80ff-4933-8b8d-97f26e582ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:13:27.979603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559361f-80ff-4933-8b8d-97f26e582ff3 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:13:27.979880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559361f-80ff-4933-8b8d-97f26e582ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:13:27.980147 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9559361f-80ff-4933-8b8d-97f26e582ff3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:13:27.980848 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559361f-80ff-4933-8b8d-97f26e582ff3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:13:27.982583 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: WARNING py.warnings [None req-9559361f-80ff-4933-8b8d-97f26e582ff3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 02:13:27.982583 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: warnings.warn(msg) Aug 04 02:13:27.982583 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: Aug 04 02:13:27.984337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9559361f-80ff-4933-8b8d-97f26e582ff3 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:13:28.039902 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 754/1497] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:13:27 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:13:28.044321 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f64641-c67a-42ab-8a7b-5dbbdac7ed81 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:13:28.045887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-05f64641-c67a-42ab-8a7b-5dbbdac7ed81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:13:28.081332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f64641-c67a-42ab-8a7b-5dbbdac7ed81 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:13:28.179095 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c37ebb-fe99-419d-9e82-3bccf2ab1cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:13:28.179703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c37ebb-fe99-419d-9e82-3bccf2ab1cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:13:28.179972 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c37ebb-fe99-419d-9e82-3bccf2ab1cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:13:28.180238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c37ebb-fe99-419d-9e82-3bccf2ab1cc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:13:28.180913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c37ebb-fe99-419d-9e82-3bccf2ab1cc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:13:28.181914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: WARNING py.warnings [None req-26c37ebb-fe99-419d-9e82-3bccf2ab1cc2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 02:13:28.181914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: warnings.warn(msg) Aug 04 02:13:28.181914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: Aug 04 02:13:28.183116 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c37ebb-fe99-419d-9e82-3bccf2ab1cc2 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:13:28.224648 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 744/1498] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:13:28 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:13:28.228686 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559361f-80ff-4933-8b8d-97f26e582ff3 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:13:28.230170 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9559361f-80ff-4933-8b8d-97f26e582ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:13:28.292714 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9559361f-80ff-4933-8b8d-97f26e582ff3 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:13:28.384219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:13:28.395455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8dc49-356c-4757-8d5d-f395a77da4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:13:28.396074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8dc49-356c-4757-8d5d-f395a77da4b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:13:28.396348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8dc49-356c-4757-8d5d-f395a77da4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:13:28.396612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed8dc49-356c-4757-8d5d-f395a77da4b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:13:28.397363 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8dc49-356c-4757-8d5d-f395a77da4b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:13:28.398384 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: WARNING py.warnings [None req-2ed8dc49-356c-4757-8d5d-f395a77da4b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 02:13:28.398384 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: warnings.warn(msg) Aug 04 02:13:28.398384 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: Aug 04 02:13:28.399278 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed8dc49-356c-4757-8d5d-f395a77da4b4 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:13:28.412662 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 755/1499] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:13:28.416892 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c37ebb-fe99-419d-9e82-3bccf2ab1cc2 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:13:28.418569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-26c37ebb-fe99-419d-9e82-3bccf2ab1cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:13:28.430931 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c37ebb-fe99-419d-9e82-3bccf2ab1cc2 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:13:28.457291 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:13:28.528509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926c235-2bc4-459d-951d-16c2a1ab4148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:13:28.529061 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926c235-2bc4-459d-951d-16c2a1ab4148 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:13:28.529337 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926c235-2bc4-459d-951d-16c2a1ab4148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:13:28.529605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926c235-2bc4-459d-951d-16c2a1ab4148 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:13:28.530296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926c235-2bc4-459d-951d-16c2a1ab4148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:13:28.531331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: WARNING py.warnings [None req-2926c235-2bc4-459d-951d-16c2a1ab4148 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 02:13:28.531331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: warnings.warn(msg) Aug 04 02:13:28.531331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: Aug 04 02:13:28.532219 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2926c235-2bc4-459d-951d-16c2a1ab4148 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:13:28.544380 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 745/1500] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:13:28.549291 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8dc49-356c-4757-8d5d-f395a77da4b4 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:13:28.550816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-2ed8dc49-356c-4757-8d5d-f395a77da4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:13:28.584711 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed8dc49-356c-4757-8d5d-f395a77da4b4 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:13:28.680499 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ae35b-1e2e-498f-b276-3f072468424f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:13:28.681221 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ae35b-1e2e-498f-b276-3f072468424f admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:13:28.681502 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ae35b-1e2e-498f-b276-3f072468424f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:13:28.681777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ae35b-1e2e-498f-b276-3f072468424f admin admin] PATH_INFO: `/v3/projects/25b611b001df4497b56db196ddc427b0/users/077c64c9e04b4dee829ed05cc1ddfb5c/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:13:28.703574 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ae35b-1e2e-498f-b276-3f072468424f admin admin] RBAC: Authorizing `identity:create_grant(project_id=25b611b001df4497b56db196ddc427b0, user_id=077c64c9e04b4dee829ed05cc1ddfb5c, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:13:28.704597 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: WARNING py.warnings [None req-006ae35b-1e2e-498f-b276-3f072468424f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 02:13:28.704597 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: warnings.warn(msg) Aug 04 02:13:28.704597 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: Aug 04 02:13:28.706584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ae35b-1e2e-498f-b276-3f072468424f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:13:28.732084 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 756/1501] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:13:28 2022] PUT /identity/v3/projects/25b611b001df4497b56db196ddc427b0/users/077c64c9e04b4dee829ed05cc1ddfb5c/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:13:28.744966 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c517b-ae15-4a48-b607-a1f1a2f0f116 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:13:28.745246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c517b-ae15-4a48-b607-a1f1a2f0f116 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:13:28.745542 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c517b-ae15-4a48-b607-a1f1a2f0f116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:13:28.804924 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-dd1c517b-ae15-4a48-b607-a1f1a2f0f116 None None] MFA Rules not processed for user `077c64c9e04b4dee829ed05cc1ddfb5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:13:28.806209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-dd1c517b-ae15-4a48-b607-a1f1a2f0f116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:13:28.887263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 746/1502] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:13:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:13:28.900443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a14623-2ee0-45a0-bd93-2d9416843b42 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:13:28.900759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a14623-2ee0-45a0-bd93-2d9416843b42 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:13:28.901043 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a14623-2ee0-45a0-bd93-2d9416843b42 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:13:28.902448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 757/1503] 10.222.0.45 () {54 vars in 964 bytes} [Thu Aug 4 02:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:13:28.908302 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf3250-2f50-4d7d-94ea-7c49fdb2f324 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:13:28.908588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf3250-2f50-4d7d-94ea-7c49fdb2f324 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:13:28.908930 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf3250-2f50-4d7d-94ea-7c49fdb2f324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:13:28.941039 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:13:28.986099 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-eeaf3250-2f50-4d7d-94ea-7c49fdb2f324 None None] MFA Rules not processed for user `f4b8838129174fbb8cdd32bd5d4d6101`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:13:28.987302 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-eeaf3250-2f50-4d7d-94ea-7c49fdb2f324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:13:29.062314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 747/1504] 10.222.0.45 () {58 vars in 1061 bytes} [Thu Aug 4 02:13:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:13:29.070046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a14623-2ee0-45a0-bd93-2d9416843b42 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:13:29.071553 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d1a14623-2ee0-45a0-bd93-2d9416843b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:13:29.123171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:13:29.177423 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a14623-2ee0-45a0-bd93-2d9416843b42 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:13:29.248292 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4966dfd-eadf-4ac0-b4ca-b1be42daeaff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:13:29.248920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4966dfd-eadf-4ac0-b4ca-b1be42daeaff service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:13:29.249197 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4966dfd-eadf-4ac0-b4ca-b1be42daeaff service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:13:29.249494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4966dfd-eadf-4ac0-b4ca-b1be42daeaff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:13:29.251112 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b4966dfd-eadf-4ac0-b4ca-b1be42daeaff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:13:29.350731 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4966dfd-eadf-4ac0-b4ca-b1be42daeaff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:13:29.352224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4966dfd-eadf-4ac0-b4ca-b1be42daeaff service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:13:29.357514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:13:29.373401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 758/1505] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:13:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:13:29.418678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbcebc-e365-4b2a-b8dc-aa0fd6aaf4a6 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:13:29.419100 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbcebc-e365-4b2a-b8dc-aa0fd6aaf4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:13:29.419272 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bbcebc-e365-4b2a-b8dc-aa0fd6aaf4a6 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:13:29.420538 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 748/1506] 10.222.0.45 () {52 vars in 830 bytes} [Thu Aug 4 02:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:13:29.424226 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4966dfd-eadf-4ac0-b4ca-b1be42daeaff service cinder] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:13:29.425510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b4966dfd-eadf-4ac0-b4ca-b1be42daeaff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:13:29.524679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4966dfd-eadf-4ac0-b4ca-b1be42daeaff service cinder] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:13:29.617380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7c8a0-ce76-4969-be6b-0f7588c21f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:13:29.618034 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7c8a0-ce76-4969-be6b-0f7588c21f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:13:29.618193 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7c8a0-ce76-4969-be6b-0f7588c21f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:13:29.618466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7c8a0-ce76-4969-be6b-0f7588c21f88 admin admin] PATH_INFO: `/v3/users/077c64c9e04b4dee829ed05cc1ddfb5c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:13:29.631187 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7c8a0-ce76-4969-be6b-0f7588c21f88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=077c64c9e04b4dee829ed05cc1ddfb5c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:13:29.632205 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: WARNING py.warnings [None req-95b7c8a0-ce76-4969-be6b-0f7588c21f88 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 02:13:29.632205 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: warnings.warn(msg) Aug 04 02:13:29.632205 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: Aug 04 02:13:29.633461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7c8a0-ce76-4969-be6b-0f7588c21f88 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:13:29.643707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:13:29.700196 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-95b7c8a0-ce76-4969-be6b-0f7588c21f88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '077c64c9e04b4dee829ed05cc1ddfb5c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:13:29.715082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-95b7c8a0-ce76-4969-be6b-0f7588c21f88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '077c64c9e04b4dee829ed05cc1ddfb5c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:13:29.715647 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-95b7c8a0-ce76-4969-be6b-0f7588c21f88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '077c64c9e04b4dee829ed05cc1ddfb5c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:13:29.738433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-95b7c8a0-ce76-4969-be6b-0f7588c21f88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '077c64c9e04b4dee829ed05cc1ddfb5c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:13:29.738879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-95b7c8a0-ce76-4969-be6b-0f7588c21f88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '077c64c9e04b4dee829ed05cc1ddfb5c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:13:29.753890 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 759/1507] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:13:29 2022] DELETE /identity/v3/users/077c64c9e04b4dee829ed05cc1ddfb5c => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:13:29.758295 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b2f38-9475-443b-bf3d-fbc7384b76a6 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:13:29.758583 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b2f38-9475-443b-bf3d-fbc7384b76a6 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:13:29.758867 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b2f38-9475-443b-bf3d-fbc7384b76a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:13:29.821898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-092b2f38-9475-443b-bf3d-fbc7384b76a6 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:13:29.823113 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-092b2f38-9475-443b-bf3d-fbc7384b76a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:13:29.899993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:13:29.914978 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 749/1508] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:13:29.925310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7c8a0-ce76-4969-be6b-0f7588c21f88 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:13:29.926528 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-95b7c8a0-ce76-4969-be6b-0f7588c21f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:13:30.026155 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7c8a0-ce76-4969-be6b-0f7588c21f88 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:13:30.081428 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:13:30.103328 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c1306-4893-498c-80bf-29d7b89aa21a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:13:30.103971 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c1306-4893-498c-80bf-29d7b89aa21a service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:13:30.104250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c1306-4893-498c-80bf-29d7b89aa21a service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:13:30.104547 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c1306-4893-498c-80bf-29d7b89aa21a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:13:30.106193 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-2c6c1306-4893-498c-80bf-29d7b89aa21a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:13:30.213026 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c1306-4893-498c-80bf-29d7b89aa21a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:13:30.214321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c1306-4893-498c-80bf-29d7b89aa21a service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:13:30.233109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 760/1509] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:13:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:13:30.587387 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b2f38-9475-443b-bf3d-fbc7384b76a6 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:13:30.588610 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-092b2f38-9475-443b-bf3d-fbc7384b76a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:13:30.600711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b2f38-9475-443b-bf3d-fbc7384b76a6 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:13:30.696757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc435b56-33fb-4b7a-9f18-988fea06d0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:13:30.697399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc435b56-33fb-4b7a-9f18-988fea06d0e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:13:30.697681 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc435b56-33fb-4b7a-9f18-988fea06d0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:13:30.697954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc435b56-33fb-4b7a-9f18-988fea06d0e3 admin admin] PATH_INFO: `/v3/projects/25b611b001df4497b56db196ddc427b0` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:13:30.708378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc435b56-33fb-4b7a-9f18-988fea06d0e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25b611b001df4497b56db196ddc427b0)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:13:30.709437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: WARNING py.warnings [None req-dc435b56-33fb-4b7a-9f18-988fea06d0e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 02:13:30.709437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: warnings.warn(msg) Aug 04 02:13:30.709437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: Aug 04 02:13:30.710641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc435b56-33fb-4b7a-9f18-988fea06d0e3 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:13:30.767490 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-dc435b56-33fb-4b7a-9f18-988fea06d0e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25b611b001df4497b56db196ddc427b0', 'request_id': 'req-dc435b56-33fb-4b7a-9f18-988fea06d0e3'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:13:30.777945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-dc435b56-33fb-4b7a-9f18-988fea06d0e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25b611b001df4497b56db196ddc427b0', 'request_id': 'req-dc435b56-33fb-4b7a-9f18-988fea06d0e3'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:13:30.786533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-dc435b56-33fb-4b7a-9f18-988fea06d0e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25b611b001df4497b56db196ddc427b0', 'request_id': 'req-dc435b56-33fb-4b7a-9f18-988fea06d0e3'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:13:30.815168 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-dc435b56-33fb-4b7a-9f18-988fea06d0e3 admin admin] The token cache is being invalidate because project 25b611b001df4497b56db196ddc427b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:13:30.815458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-dc435b56-33fb-4b7a-9f18-988fea06d0e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:13:30.815891 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-dc435b56-33fb-4b7a-9f18-988fea06d0e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:13:30.817340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 750/1510] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:13:30 2022] DELETE /identity/v3/projects/25b611b001df4497b56db196ddc427b0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:13:40.230782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=62 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:13:40.813243 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:14:57.540895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0bb7f-8e57-4e20-b57d-fc6eaa68aaae None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:14:57.541510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0bb7f-8e57-4e20-b57d-fc6eaa68aaae None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:14:57.541510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe0bb7f-8e57-4e20-b57d-fc6eaa68aaae None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:14:57.542810 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 761/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 02:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:14:57.547900 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59b9e6-38fe-45fc-868d-f3ae9dc85f3f None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:14:57.548334 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59b9e6-38fe-45fc-868d-f3ae9dc85f3f None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:14:57.548453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59b9e6-38fe-45fc-868d-f3ae9dc85f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:14:57.630622 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-bb59b9e6-38fe-45fc-868d-f3ae9dc85f3f None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:14:57.631851 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-bb59b9e6-38fe-45fc-868d-f3ae9dc85f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:14:57.738509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 751/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 02:14:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:14:57.896563 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0bb7f-8e57-4e20-b57d-fc6eaa68aaae None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:14:57.898756 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-bbe0bb7f-8e57-4e20-b57d-fc6eaa68aaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:14:58.022970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe0bb7f-8e57-4e20-b57d-fc6eaa68aaae None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:14:58.028340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4100da-1e03-4948-9b2b-9c6f7b6470d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:14:58.028980 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4100da-1e03-4948-9b2b-9c6f7b6470d0 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:14:58.029264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4100da-1e03-4948-9b2b-9c6f7b6470d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:14:58.029538 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4100da-1e03-4948-9b2b-9c6f7b6470d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:14:58.037700 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4100da-1e03-4948-9b2b-9c6f7b6470d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:14:58.039274 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4100da-1e03-4948-9b2b-9c6f7b6470d0 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:14:58.048561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 762/1513] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:14:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:14:58.890838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2087c88-f86f-4b94-876a-3b3cd2ae7261 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:14:58.891427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2087c88-f86f-4b94-876a-3b3cd2ae7261 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:14:58.891427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2087c88-f86f-4b94-876a-3b3cd2ae7261 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:14:58.892780 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 752/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 02:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:14:58.898321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9115e46-a4c3-4bb0-9c87-084c5b19d4c9 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:14:58.898749 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9115e46-a4c3-4bb0-9c87-084c5b19d4c9 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:14:58.898858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9115e46-a4c3-4bb0-9c87-084c5b19d4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:14:58.921858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-e9115e46-a4c3-4bb0-9c87-084c5b19d4c9 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:14:58.922944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e9115e46-a4c3-4bb0-9c87-084c5b19d4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:14:58.930729 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 763/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 02:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:14:59.083763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2087c88-f86f-4b94-876a-3b3cd2ae7261 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:14:59.098038 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2087c88-f86f-4b94-876a-3b3cd2ae7261 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:14:59.103399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efaa22b-8db0-4609-b2c0-bc006a69967f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:14:59.104028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaa22b-8db0-4609-b2c0-bc006a69967f service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:14:59.104309 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaa22b-8db0-4609-b2c0-bc006a69967f service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:14:59.104583 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaa22b-8db0-4609-b2c0-bc006a69967f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:14:59.112726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efaa22b-8db0-4609-b2c0-bc006a69967f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:14:59.114345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efaa22b-8db0-4609-b2c0-bc006a69967f service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:14:59.123649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 753/1516] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:14:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:15:00.654087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659634f-bddc-46f2-86cf-d2fcd28a8597 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:00.654790 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659634f-bddc-46f2-86cf-d2fcd28a8597 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:00.654790 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659634f-bddc-46f2-86cf-d2fcd28a8597 None None] PATH_INFO: `/v3` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:00.655930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 764/1517] 10.222.0.36 () {54 vars in 925 bytes} [Thu Aug 4 02:15:00 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:15:00.663924 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a99b62-1b5b-4b9a-9c31-e6bd262b6de3 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:00.664332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a99b62-1b5b-4b9a-9c31-e6bd262b6de3 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:00.664472 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a99b62-1b5b-4b9a-9c31-e6bd262b6de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:00.722754 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-a6a99b62-1b5b-4b9a-9c31-e6bd262b6de3 None None] MFA Rules not processed for user `796772cf491047c8b0a609dae9402307`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:15:00.723938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a6a99b62-1b5b-4b9a-9c31-e6bd262b6de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:15:00.740037 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:15:00.795535 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 754/1518] 10.222.0.36 () {58 vars in 1015 bytes} [Thu Aug 4 02:15:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:15:04.713715 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc222215-2694-4506-b39e-844359400091 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:04.714445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc222215-2694-4506-b39e-844359400091 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:04.714445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc222215-2694-4506-b39e-844359400091 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:04.715586 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 765/1519] 10.222.0.45 () {54 vars in 961 bytes} [Thu Aug 4 02:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:15:04.722452 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02368f2d-0731-4d2c-9c64-f6ebfb9fc653 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:04.722902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02368f2d-0731-4d2c-9c64-f6ebfb9fc653 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:04.723013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02368f2d-0731-4d2c-9c64-f6ebfb9fc653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:04.742904 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-02368f2d-0731-4d2c-9c64-f6ebfb9fc653 None None] MFA Rules not processed for user `796772cf491047c8b0a609dae9402307`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:15:04.744093 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-02368f2d-0731-4d2c-9c64-f6ebfb9fc653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:15:04.753215 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 755/1520] 10.222.0.45 () {58 vars in 1058 bytes} [Thu Aug 4 02:15:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:15:04.760632 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc222215-2694-4506-b39e-844359400091 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:15:04.773600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc222215-2694-4506-b39e-844359400091 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:15:04.778427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e09e2f-77ec-49bd-b2a0-c6ef866793c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796772cf491047c8b0a609dae9402307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:15:04.779017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e09e2f-77ec-49bd-b2a0-c6ef866793c2 service placement] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:04.779312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e09e2f-77ec-49bd-b2a0-c6ef866793c2 service placement] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:04.779588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e09e2f-77ec-49bd-b2a0-c6ef866793c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:04.787820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e09e2f-77ec-49bd-b2a0-c6ef866793c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:15:04.789425 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e09e2f-77ec-49bd-b2a0-c6ef866793c2 service placement] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:15:04.799800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 766/1521] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:15:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:15:10.368888 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ba5cc-c9a8-44a7-9024-1f0807a9f36f None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:10.369389 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ba5cc-c9a8-44a7-9024-1f0807a9f36f None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:10.369501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ba5cc-c9a8-44a7-9024-1f0807a9f36f None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:10.370798 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 02:15:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:15:10.375668 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c186391-88d0-494f-b587-a5f70b3490c4 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:10.376071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c186391-88d0-494f-b587-a5f70b3490c4 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:10.376346 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c186391-88d0-494f-b587-a5f70b3490c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:10.397021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-2c186391-88d0-494f-b587-a5f70b3490c4 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:15:10.398257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-2c186391-88d0-494f-b587-a5f70b3490c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:15:10.406179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 02:15:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:15:10.560874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ba5cc-c9a8-44a7-9024-1f0807a9f36f None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:15:10.575944 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ba5cc-c9a8-44a7-9024-1f0807a9f36f None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:15:10.581604 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c3190-e37f-48c6-8bc0-e83ed5b08817 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:15:10.582212 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c3190-e37f-48c6-8bc0-e83ed5b08817 service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:10.582485 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c3190-e37f-48c6-8bc0-e83ed5b08817 service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:10.582757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c3190-e37f-48c6-8bc0-e83ed5b08817 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:10.590745 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c3190-e37f-48c6-8bc0-e83ed5b08817 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:15:10.592312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c3190-e37f-48c6-8bc0-e83ed5b08817 service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:15:10.601588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 757/1524] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:15:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:15:11.426311 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac761c-9df3-47fd-bc22-e98b4d780088 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:11.426896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac761c-9df3-47fd-bc22-e98b4d780088 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:11.426896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ac761c-9df3-47fd-bc22-e98b4d780088 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:11.428227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Aug 4 02:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:15:11.433001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c113d-ee2e-4677-b8d1-10f1b52c7777 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:11.433436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c113d-ee2e-4677-b8d1-10f1b52c7777 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:11.433533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c113d-ee2e-4677-b8d1-10f1b52c7777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:11.453498 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-8f9c113d-ee2e-4677-b8d1-10f1b52c7777 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:15:11.454714 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-8f9c113d-ee2e-4677-b8d1-10f1b52c7777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:15:11.463251 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Aug 4 02:15:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:15:11.617245 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac761c-9df3-47fd-bc22-e98b4d780088 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:15:11.631451 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ac761c-9df3-47fd-bc22-e98b4d780088 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:15:11.637314 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152cf5f5-44c1-4784-bbb5-82e892c069d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:15:11.637914 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cf5f5-44c1-4784-bbb5-82e892c069d1 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:11.638432 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cf5f5-44c1-4784-bbb5-82e892c069d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:11.638432 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152cf5f5-44c1-4784-bbb5-82e892c069d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:11.647541 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cf5f5-44c1-4784-bbb5-82e892c069d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:15:11.649173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152cf5f5-44c1-4784-bbb5-82e892c069d1 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:15:11.659488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 769/1527] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:15:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:15:15.711607 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88a294-3ca3-4ac8-b283-97272885636c None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:15.712129 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88a294-3ca3-4ac8-b283-97272885636c None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:15.712129 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88a294-3ca3-4ac8-b283-97272885636c None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:15.713463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 02:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:15:15.718601 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95203b08-3de2-49b2-9e85-1db296530499 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:15.719029 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95203b08-3de2-49b2-9e85-1db296530499 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:15.719132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95203b08-3de2-49b2-9e85-1db296530499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:15.738172 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-95203b08-3de2-49b2-9e85-1db296530499 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:15:15.739349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-95203b08-3de2-49b2-9e85-1db296530499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:15:15.747089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 02:15:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:15:15.820830 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88a294-3ca3-4ac8-b283-97272885636c None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:15:15.822529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ca88a294-3ca3-4ac8-b283-97272885636c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:15:15.899920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88a294-3ca3-4ac8-b283-97272885636c None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:15:15.903481 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b970ff2-2b42-4630-8b29-e0e9fbb07cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:15:15.904101 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b970ff2-2b42-4630-8b29-e0e9fbb07cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:15.904380 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b970ff2-2b42-4630-8b29-e0e9fbb07cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:15.904686 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b970ff2-2b42-4630-8b29-e0e9fbb07cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:15.914050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b970ff2-2b42-4630-8b29-e0e9fbb07cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:15:15.915472 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b970ff2-2b42-4630-8b29-e0e9fbb07cf8 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:15:15.926408 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 760/1530] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:15:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:15:17.951692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a097e1-1fe1-42c9-a6e5-fa50e7702b90 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:17.952552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a097e1-1fe1-42c9-a6e5-fa50e7702b90 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:17.952552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a097e1-1fe1-42c9-a6e5-fa50e7702b90 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:17.953877 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 02:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:15:17.959070 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45a7c3-1a15-4103-a0db-63cb1675e5aa None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:17.959556 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45a7c3-1a15-4103-a0db-63cb1675e5aa None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:17.960084 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45a7c3-1a15-4103-a0db-63cb1675e5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:17.979542 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-5c45a7c3-1a15-4103-a0db-63cb1675e5aa None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:15:17.980791 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-5c45a7c3-1a15-4103-a0db-63cb1675e5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:15:17.989492 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 02:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:15:18.025800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a097e1-1fe1-42c9-a6e5-fa50e7702b90 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:15:18.027986 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-87a097e1-1fe1-42c9-a6e5-fa50e7702b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:15:18.042342 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a097e1-1fe1-42c9-a6e5-fa50e7702b90 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:15:18.045920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09048481-31bd-420a-bdf4-7389ed89ac51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:15:18.046524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09048481-31bd-420a-bdf4-7389ed89ac51 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:18.046807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09048481-31bd-420a-bdf4-7389ed89ac51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:18.047082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09048481-31bd-420a-bdf4-7389ed89ac51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:18.056803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09048481-31bd-420a-bdf4-7389ed89ac51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:15:18.058239 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09048481-31bd-420a-bdf4-7389ed89ac51 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:15:18.067537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 772/1533] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:15:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:15:21.455277 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec1a96-0113-4cee-b2a7-b862ddfd5f93 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:21.455901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec1a96-0113-4cee-b2a7-b862ddfd5f93 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:21.455901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eec1a96-0113-4cee-b2a7-b862ddfd5f93 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:21.457238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 02:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:15:21.462376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fac975-3e6e-48bb-adef-e12053805fed None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:21.462827 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fac975-3e6e-48bb-adef-e12053805fed None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:21.462937 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fac975-3e6e-48bb-adef-e12053805fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:21.472207 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=81 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:15:21.485605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-74fac975-3e6e-48bb-adef-e12053805fed None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:15:21.487431 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-74fac975-3e6e-48bb-adef-e12053805fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:15:21.495931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 02:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:15:21.802021 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec1a96-0113-4cee-b2a7-b862ddfd5f93 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:15:21.803835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7eec1a96-0113-4cee-b2a7-b862ddfd5f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:15:21.868085 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:15:21.881430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eec1a96-0113-4cee-b2a7-b862ddfd5f93 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:15:21.885123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de72b8-dcbe-4204-a6a1-b9cb07f285ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:15:21.885734 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de72b8-dcbe-4204-a6a1-b9cb07f285ef service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:21.886009 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de72b8-dcbe-4204-a6a1-b9cb07f285ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:21.886280 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de72b8-dcbe-4204-a6a1-b9cb07f285ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:21.895573 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de72b8-dcbe-4204-a6a1-b9cb07f285ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:15:21.897029 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de72b8-dcbe-4204-a6a1-b9cb07f285ef service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:15:21.906442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 763/1536] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:15:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:15:23.949506 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2eacf-1f0b-4a2c-be6f-8bf5e7283688 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:23.950033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2eacf-1f0b-4a2c-be6f-8bf5e7283688 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:23.950033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2eacf-1f0b-4a2c-be6f-8bf5e7283688 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:23.951346 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Aug 4 02:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:15:23.956938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd2beb-03d7-4b1b-9df2-1c1cf5b932be None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:23.957402 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd2beb-03d7-4b1b-9df2-1c1cf5b932be None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:23.957507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bd2beb-03d7-4b1b-9df2-1c1cf5b932be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:23.977013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-85bd2beb-03d7-4b1b-9df2-1c1cf5b932be None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:15:23.978229 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-85bd2beb-03d7-4b1b-9df2-1c1cf5b932be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:15:23.986281 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Aug 4 02:15:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:15:24.292692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2eacf-1f0b-4a2c-be6f-8bf5e7283688 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:15:24.294451 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d2b2eacf-1f0b-4a2c-be6f-8bf5e7283688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:15:24.307250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b2eacf-1f0b-4a2c-be6f-8bf5e7283688 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:15:24.310695 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed719f-fb0d-4c6d-84c3-02b89ca0598a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:15:24.311301 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed719f-fb0d-4c6d-84c3-02b89ca0598a service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:15:24.311575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed719f-fb0d-4c6d-84c3-02b89ca0598a service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:15:24.311841 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed719f-fb0d-4c6d-84c3-02b89ca0598a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:15:24.321138 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed719f-fb0d-4c6d-84c3-02b89ca0598a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:15:24.322555 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed719f-fb0d-4c6d-84c3-02b89ca0598a service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:15:24.332700 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 775/1539] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:15:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:15:33.972922 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=11 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:15:34.328031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=8 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:48.680179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38924ef6-a8ae-4560-92f6-919f8de2fe4e None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:48.680842 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38924ef6-a8ae-4560-92f6-919f8de2fe4e None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:48.680842 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38924ef6-a8ae-4560-92f6-919f8de2fe4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:48.684264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a367d-83b5-4079-8db2-d603a1a33f15 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:48.685234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a367d-83b5-4079-8db2-d603a1a33f15 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:48.685740 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a367d-83b5-4079-8db2-d603a1a33f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:48.715797 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-38924ef6-a8ae-4560-92f6-919f8de2fe4e None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:48.716829 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-428a367d-83b5-4079-8db2-d603a1a33f15 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:48.717231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-38924ef6-a8ae-4560-92f6-919f8de2fe4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:48.718066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-428a367d-83b5-4079-8db2-d603a1a33f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:48.739339 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 776/1540] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:48.740111 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 765/1541] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:48.750163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f3cc4-b41a-49c5-ab71-1c51cca37015 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:48.750537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f3cc4-b41a-49c5-ab71-1c51cca37015 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:48.750723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f3cc4-b41a-49c5-ab71-1c51cca37015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:48.770527 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521f1f3-6204-46ca-bd07-cea53259c7c2 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:48.770888 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521f1f3-6204-46ca-bd07-cea53259c7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:48.771962 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521f1f3-6204-46ca-bd07-cea53259c7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:48.805092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-0b0f3cc4-b41a-49c5-ab71-1c51cca37015 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:48.806354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-0b0f3cc4-b41a-49c5-ab71-1c51cca37015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:48.807292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-f521f1f3-6204-46ca-bd07-cea53259c7c2 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:48.808500 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-f521f1f3-6204-46ca-bd07-cea53259c7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:48.823654 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 777/1542] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:48.825065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 766/1543] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:16:48.825373 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf237a-3b4c-4299-a6ff-34348b95869e None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:48.825666 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf237a-3b4c-4299-a6ff-34348b95869e None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:48.826032 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbf237a-3b4c-4299-a6ff-34348b95869e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:48.829766 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521f1f3-6204-46ca-bd07-cea53259c7c2 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:48.846445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521f1f3-6204-46ca-bd07-cea53259c7c2 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:48.857945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8054a4-c357-4cf4-b411-2c13a9d421ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:48.858594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8054a4-c357-4cf4-b411-2c13a9d421ec admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:48.858882 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8054a4-c357-4cf4-b411-2c13a9d421ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:48.859159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8054a4-c357-4cf4-b411-2c13a9d421ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:48.859884 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8054a4-c357-4cf4-b411-2c13a9d421ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:16:48.860191 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8054a4-c357-4cf4-b411-2c13a9d421ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:48.861904 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8054a4-c357-4cf4-b411-2c13a9d421ec admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:48.864586 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-8dbf237a-3b4c-4299-a6ff-34348b95869e None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:48.865935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8dbf237a-3b4c-4299-a6ff-34348b95869e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:48.880512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 778/1544] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:48.881517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf237a-3b4c-4299-a6ff-34348b95869e None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:48.899749 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 767/1545] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:16:48.901182 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3c029-49aa-47cc-a67d-45c56426e301 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:48.901470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3c029-49aa-47cc-a67d-45c56426e301 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:48.901748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3c029-49aa-47cc-a67d-45c56426e301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:48.909612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbf237a-3b4c-4299-a6ff-34348b95869e None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:48.922867 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808cd895-07f9-4a09-a714-88271060f460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:48.923514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cd895-07f9-4a09-a714-88271060f460 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:48.923796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cd895-07f9-4a09-a714-88271060f460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:48.924072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808cd895-07f9-4a09-a714-88271060f460 admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:48.924828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808cd895-07f9-4a09-a714-88271060f460 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:16:48.925136 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808cd895-07f9-4a09-a714-88271060f460 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:48.926154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: WARNING py.warnings [None req-808cd895-07f9-4a09-a714-88271060f460 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 04 02:16:48.926154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: warnings.warn(msg) Aug 04 02:16:48.926154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: Aug 04 02:16:48.927048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808cd895-07f9-4a09-a714-88271060f460 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:48.948300 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-2dc3c029-49aa-47cc-a67d-45c56426e301 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:48.949620 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2dc3c029-49aa-47cc-a67d-45c56426e301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:48.952437 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 779/1546] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:16:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:16:48.953365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808cd895-07f9-4a09-a714-88271060f460 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:48.974448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808cd895-07f9-4a09-a714-88271060f460 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:48.980264 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 768/1547] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:48.981476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ac3d6-6d1e-45c8-adb9-864b156ffedc None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:48.981768 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ac3d6-6d1e-45c8-adb9-864b156ffedc None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:48.982049 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ac3d6-6d1e-45c8-adb9-864b156ffedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:48.986990 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbe907-0489-4ec7-9b47-8a57ffc21dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:48.987630 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbe907-0489-4ec7-9b47-8a57ffc21dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:48.988069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbe907-0489-4ec7-9b47-8a57ffc21dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:48.988446 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbe907-0489-4ec7-9b47-8a57ffc21dd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:48.989415 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbe907-0489-4ec7-9b47-8a57ffc21dd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:48.992097 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbe907-0489-4ec7-9b47-8a57ffc21dd4 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.008439 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-ff3ac3d6-6d1e-45c8-adb9-864b156ffedc None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:49.009684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ff3ac3d6-6d1e-45c8-adb9-864b156ffedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:49.028095 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 769/1548] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:49.035703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9e5d1-fc42-40b7-aa58-1c102e9455ce None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.035703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9e5d1-fc42-40b7-aa58-1c102e9455ce None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.035703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d9e5d1-fc42-40b7-aa58-1c102e9455ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.065525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-85d9e5d1-fc42-40b7-aa58-1c102e9455ce None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:49.066972 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-85d9e5d1-fc42-40b7-aa58-1c102e9455ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:49.080452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 780/1549] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:16:48 2022] POST /identity/v3/projects => generated 379 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:49.081613 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbe907-0489-4ec7-9b47-8a57ffc21dd4 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.083441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 770/1550] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:49.084430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9e5d1-fc42-40b7-aa58-1c102e9455ce None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.098429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d9e5d1-fc42-40b7-aa58-1c102e9455ce None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.161704 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbe907-0489-4ec7-9b47-8a57ffc21dd4 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.166835 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a949b-d94e-4058-967c-dd718d4eff65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.167457 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a949b-d94e-4058-967c-dd718d4eff65 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.167740 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a949b-d94e-4058-967c-dd718d4eff65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.168017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a949b-d94e-4058-967c-dd718d4eff65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.168750 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a949b-d94e-4058-967c-dd718d4eff65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.170658 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a949b-d94e-4058-967c-dd718d4eff65 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.171087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f48ae-d6ca-4e9e-9fa4-98fb3a964905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.171705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f48ae-d6ca-4e9e-9fa4-98fb3a964905 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.171985 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f48ae-d6ca-4e9e-9fa4-98fb3a964905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.172256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f48ae-d6ca-4e9e-9fa4-98fb3a964905 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.173040 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f48ae-d6ca-4e9e-9fa4-98fb3a964905 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:16:49.173281 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f48ae-d6ca-4e9e-9fa4-98fb3a964905 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.175148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f48ae-d6ca-4e9e-9fa4-98fb3a964905 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.186521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 771/1551] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:16:49.190053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae434841-45cb-4cf2-95fd-6e018f89b5f8 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.190550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae434841-45cb-4cf2-95fd-6e018f89b5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.190550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae434841-45cb-4cf2-95fd-6e018f89b5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.203807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 781/1552] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/projects => generated 391 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:49.203807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f41369-7545-4622-8d6f-451505d58e9b None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.203807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f41369-7545-4622-8d6f-451505d58e9b None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.203807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f41369-7545-4622-8d6f-451505d58e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.211253 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-ae434841-45cb-4cf2-95fd-6e018f89b5f8 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:49.212455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ae434841-45cb-4cf2-95fd-6e018f89b5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:49.223287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-99f41369-7545-4622-8d6f-451505d58e9b None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:49.224490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-99f41369-7545-4622-8d6f-451505d58e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:49.273181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 772/1553] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:49.274539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a7d40-2c5b-41f2-b38d-1d09a3ba33d7 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.274838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a7d40-2c5b-41f2-b38d-1d09a3ba33d7 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.275114 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3a7d40-2c5b-41f2-b38d-1d09a3ba33d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.284233 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 782/1554] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:49.285321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f41369-7545-4622-8d6f-451505d58e9b None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.296059 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-7d3a7d40-2c5b-41f2-b38d-1d09a3ba33d7 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:49.297343 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7d3a7d40-2c5b-41f2-b38d-1d09a3ba33d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:49.297479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f41369-7545-4622-8d6f-451505d58e9b None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.302567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfee345-ac65-467b-8a52-340781cc7a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.303188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfee345-ac65-467b-8a52-340781cc7a9e admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.303471 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfee345-ac65-467b-8a52-340781cc7a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.304260 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfee345-ac65-467b-8a52-340781cc7a9e admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.304964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfee345-ac65-467b-8a52-340781cc7a9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.305975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: WARNING py.warnings [None req-1dfee345-ac65-467b-8a52-340781cc7a9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 02:16:49.305975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: warnings.warn(msg) Aug 04 02:16:49.305975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: Aug 04 02:16:49.306295 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 773/1555] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:49.307270 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfee345-ac65-467b-8a52-340781cc7a9e admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.307458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603ed85-94f7-45d4-bccd-c6c10e209c1a None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.307738 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603ed85-94f7-45d4-bccd-c6c10e209c1a None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.308011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603ed85-94f7-45d4-bccd-c6c10e209c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.327849 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-a603ed85-94f7-45d4-bccd-c6c10e209c1a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:49.329072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a603ed85-94f7-45d4-bccd-c6c10e209c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:49.330675 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 783/1556] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/users => generated 290 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:49.331896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e6a9d-2f11-4179-ae12-2ab213cb03d1 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.332188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e6a9d-2f11-4179-ae12-2ab213cb03d1 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.332467 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e6a9d-2f11-4179-ae12-2ab213cb03d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.337338 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 774/1557] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:49.338089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603ed85-94f7-45d4-bccd-c6c10e209c1a None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.342609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:49.347532 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603ed85-94f7-45d4-bccd-c6c10e209c1a None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.353133 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a3e06-e842-488d-a7a1-bf7aeace2c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.353778 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a3e06-e842-488d-a7a1-bf7aeace2c82 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.354092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a3e06-e842-488d-a7a1-bf7aeace2c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.354364 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a3e06-e842-488d-a7a1-bf7aeace2c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.355026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a3e06-e842-488d-a7a1-bf7aeace2c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.355741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-6f4e6a9d-2f11-4179-ae12-2ab213cb03d1 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:49.356045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: WARNING py.warnings [None req-130a3e06-e842-488d-a7a1-bf7aeace2c82 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 02:16:49.356045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: warnings.warn(msg) Aug 04 02:16:49.356045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: Aug 04 02:16:49.356984 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-6f4e6a9d-2f11-4179-ae12-2ab213cb03d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:49.357296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a3e06-e842-488d-a7a1-bf7aeace2c82 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.365671 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 784/1558] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:49.367079 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e6a9d-2f11-4179-ae12-2ab213cb03d1 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.379477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e6a9d-2f11-4179-ae12-2ab213cb03d1 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.385967 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9756a5f-0ccc-45dc-a1bf-982ac036823b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.386593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9756a5f-0ccc-45dc-a1bf-982ac036823b admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.387027 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9756a5f-0ccc-45dc-a1bf-982ac036823b admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.387370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9756a5f-0ccc-45dc-a1bf-982ac036823b admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.388391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9756a5f-0ccc-45dc-a1bf-982ac036823b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.390570 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9756a5f-0ccc-45dc-a1bf-982ac036823b admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.404526 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 775/1559] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/projects => generated 363 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:49.405667 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a3e06-e842-488d-a7a1-bf7aeace2c82 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.420786 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 785/1560] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/users => generated 296 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:49.421221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:49.422229 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e99ebe-0541-463d-afe9-9b69973aeac3 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.422699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e99ebe-0541-463d-afe9-9b69973aeac3 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.423012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e99ebe-0541-463d-afe9-9b69973aeac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.443940 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-00e99ebe-0541-463d-afe9-9b69973aeac3 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:49.445762 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-00e99ebe-0541-463d-afe9-9b69973aeac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:49.475399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a3e06-e842-488d-a7a1-bf7aeace2c82 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.480812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1dd631-cef3-467f-b7b7-f4b2c9490f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.481441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dd631-cef3-467f-b7b7-f4b2c9490f74 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.481725 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dd631-cef3-467f-b7b7-f4b2c9490f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.482001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1dd631-cef3-467f-b7b7-f4b2c9490f74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.482704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1dd631-cef3-467f-b7b7-f4b2c9490f74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:16:49.483008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1dd631-cef3-467f-b7b7-f4b2c9490f74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.484610 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1dd631-cef3-467f-b7b7-f4b2c9490f74 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.496463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 776/1561] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:16:49.497764 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dd083-4f42-416d-bb60-5c2effbc1ecd None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.498035 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dd083-4f42-416d-bb60-5c2effbc1ecd None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.498865 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dd083-4f42-416d-bb60-5c2effbc1ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.508982 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 786/1562] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:49.509777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e99ebe-0541-463d-afe9-9b69973aeac3 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.518984 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e99ebe-0541-463d-afe9-9b69973aeac3 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.520172 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-6f1dd083-4f42-416d-bb60-5c2effbc1ecd None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:49.521396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-6f1dd083-4f42-416d-bb60-5c2effbc1ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:49.524544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add97855-9e0f-4618-b07f-a44254daf442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.525190 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add97855-9e0f-4618-b07f-a44254daf442 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.525473 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add97855-9e0f-4618-b07f-a44254daf442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.525753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add97855-9e0f-4618-b07f-a44254daf442 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.526605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add97855-9e0f-4618-b07f-a44254daf442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.528246 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add97855-9e0f-4618-b07f-a44254daf442 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.529797 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 777/1563] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:49.533592 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1dd083-4f42-416d-bb60-5c2effbc1ecd None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.542206 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 787/1564] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:49.542388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1dd083-4f42-416d-bb60-5c2effbc1ecd None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.542486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910bbe3-99c8-4825-8082-f9f6ea1b965e None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.542486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910bbe3-99c8-4825-8082-f9f6ea1b965e None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.542486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910bbe3-99c8-4825-8082-f9f6ea1b965e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.544502 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75cf1a-68c1-4a97-9a5a-8911ceeb803e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.545143 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75cf1a-68c1-4a97-9a5a-8911ceeb803e admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.545423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75cf1a-68c1-4a97-9a5a-8911ceeb803e admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.545694 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75cf1a-68c1-4a97-9a5a-8911ceeb803e admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.546375 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75cf1a-68c1-4a97-9a5a-8911ceeb803e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:16:49.546719 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75cf1a-68c1-4a97-9a5a-8911ceeb803e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.548275 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75cf1a-68c1-4a97-9a5a-8911ceeb803e admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.565671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 778/1565] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:16:49.565671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75cf1a-68c1-4a97-9a5a-8911ceeb803e admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.565981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-4910bbe3-99c8-4825-8082-f9f6ea1b965e None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:49.565981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4910bbe3-99c8-4825-8082-f9f6ea1b965e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:49.572049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 788/1566] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:49.572507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75cf1a-68c1-4a97-9a5a-8911ceeb803e admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.572868 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910bbe3-99c8-4825-8082-f9f6ea1b965e None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.577085 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679a671-1d39-4993-960e-f0bd03abdca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.577817 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679a671-1d39-4993-960e-f0bd03abdca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.578090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679a671-1d39-4993-960e-f0bd03abdca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.578361 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8679a671-1d39-4993-960e-f0bd03abdca6 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.579011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679a671-1d39-4993-960e-f0bd03abdca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.581191 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8679a671-1d39-4993-960e-f0bd03abdca6 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.583349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910bbe3-99c8-4825-8082-f9f6ea1b965e None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.589071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c5289-f30d-4d57-a3a5-5219d0d01acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.589609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c5289-f30d-4d57-a3a5-5219d0d01acd admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.589884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c5289-f30d-4d57-a3a5-5219d0d01acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.590152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c5289-f30d-4d57-a3a5-5219d0d01acd admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.590839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c5289-f30d-4d57-a3a5-5219d0d01acd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.592528 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c5289-f30d-4d57-a3a5-5219d0d01acd admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.603490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 789/1567] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:49.604352 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c5289-f30d-4d57-a3a5-5219d0d01acd admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.608666 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 779/1568] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/users => generated 282 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:49.608666 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679a671-1d39-4993-960e-f0bd03abdca6 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.613997 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c5289-f30d-4d57-a3a5-5219d0d01acd admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.617926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8679a671-1d39-4993-960e-f0bd03abdca6 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.618669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbbb07-6383-49ff-b451-48d9339b3970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.619269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbbb07-6383-49ff-b451-48d9339b3970 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.619563 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbbb07-6383-49ff-b451-48d9339b3970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.619823 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fbbb07-6383-49ff-b451-48d9339b3970 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.620481 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbbb07-6383-49ff-b451-48d9339b3970 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.622393 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fbbb07-6383-49ff-b451-48d9339b3970 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.622750 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e8c78-17f8-4dbf-b07c-3baeac0e3748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.623392 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e8c78-17f8-4dbf-b07c-3baeac0e3748 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.623676 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e8c78-17f8-4dbf-b07c-3baeac0e3748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.623943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e8c78-17f8-4dbf-b07c-3baeac0e3748 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.624658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e8c78-17f8-4dbf-b07c-3baeac0e3748 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:16:49.624965 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e8c78-17f8-4dbf-b07c-3baeac0e3748 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.626517 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30e8c78-17f8-4dbf-b07c-3baeac0e3748 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.639311 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 780/1569] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:16:49.640323 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e8c78-17f8-4dbf-b07c-3baeac0e3748 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.652910 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e8c78-17f8-4dbf-b07c-3baeac0e3748 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.657825 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33214d6-004e-4183-b867-6a1e4dc03736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.658477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33214d6-004e-4183-b867-6a1e4dc03736 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.658759 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33214d6-004e-4183-b867-6a1e4dc03736 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.659034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33214d6-004e-4183-b867-6a1e4dc03736 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.659730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33214d6-004e-4183-b867-6a1e4dc03736 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:16:49.660032 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33214d6-004e-4183-b867-6a1e4dc03736 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.661670 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33214d6-004e-4183-b867-6a1e4dc03736 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.662915 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 790/1570] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:49.663848 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbbb07-6383-49ff-b451-48d9339b3970 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.677452 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 781/1571] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:16:49.678436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33214d6-004e-4183-b867-6a1e4dc03736 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.688200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33214d6-004e-4183-b867-6a1e4dc03736 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.732019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fbbb07-6383-49ff-b451-48d9339b3970 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.738767 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47513012-0167-4a66-8d9f-fc589940c4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.739393 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47513012-0167-4a66-8d9f-fc589940c4e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.739673 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47513012-0167-4a66-8d9f-fc589940c4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.739947 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47513012-0167-4a66-8d9f-fc589940c4e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.740688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47513012-0167-4a66-8d9f-fc589940c4e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.742356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47513012-0167-4a66-8d9f-fc589940c4e8 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.747982 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a456310-d40d-4450-ae88-a4d2d11c531e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.748611 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a456310-d40d-4450-ae88-a4d2d11c531e admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.748915 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a456310-d40d-4450-ae88-a4d2d11c531e admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.749312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a456310-d40d-4450-ae88-a4d2d11c531e admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.749892 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a456310-d40d-4450-ae88-a4d2d11c531e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.751806 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a456310-d40d-4450-ae88-a4d2d11c531e admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.752549 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 791/1572] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:49.753385 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47513012-0167-4a66-8d9f-fc589940c4e8 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.763940 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47513012-0167-4a66-8d9f-fc589940c4e8 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.768645 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43847d3b-de3f-47bb-ab8e-796d5611292d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.769367 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43847d3b-de3f-47bb-ab8e-796d5611292d admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.769567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43847d3b-de3f-47bb-ab8e-796d5611292d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.769865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43847d3b-de3f-47bb-ab8e-796d5611292d admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.770552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43847d3b-de3f-47bb-ab8e-796d5611292d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:16:49.770858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43847d3b-de3f-47bb-ab8e-796d5611292d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.773100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43847d3b-de3f-47bb-ab8e-796d5611292d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.780117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 782/1573] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/projects => generated 389 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:49.781068 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a456310-d40d-4450-ae88-a4d2d11c531e admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.785567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 792/1574] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:16:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:16:49.786433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43847d3b-de3f-47bb-ab8e-796d5611292d admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.796561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43847d3b-de3f-47bb-ab8e-796d5611292d admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.806054 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:49.814227 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:49.848174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a456310-d40d-4450-ae88-a4d2d11c531e admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.853374 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123eb890-4282-408f-be70-9644d30a28fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.854023 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123eb890-4282-408f-be70-9644d30a28fb admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.854154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeab86c-18de-4c6a-b6a6-53105863336e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.854307 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123eb890-4282-408f-be70-9644d30a28fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.854580 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123eb890-4282-408f-be70-9644d30a28fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.854703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeab86c-18de-4c6a-b6a6-53105863336e admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.854929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeab86c-18de-4c6a-b6a6-53105863336e admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.855199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeab86c-18de-4c6a-b6a6-53105863336e admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.855302 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123eb890-4282-408f-be70-9644d30a28fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.855883 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeab86c-18de-4c6a-b6a6-53105863336e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.856891 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123eb890-4282-408f-be70-9644d30a28fb admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.857475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeab86c-18de-4c6a-b6a6-53105863336e admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.866807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 783/1575] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:49.867592 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123eb890-4282-408f-be70-9644d30a28fb admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.867713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 793/1576] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:49.868463 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeab86c-18de-4c6a-b6a6-53105863336e admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.877013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123eb890-4282-408f-be70-9644d30a28fb admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.881125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeab86c-18de-4c6a-b6a6-53105863336e admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.881633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a842d79-13b0-41cb-9a4f-bdab434f901a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.882265 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a842d79-13b0-41cb-9a4f-bdab434f901a admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.882545 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a842d79-13b0-41cb-9a4f-bdab434f901a admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.882819 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a842d79-13b0-41cb-9a4f-bdab434f901a admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.883477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a842d79-13b0-41cb-9a4f-bdab434f901a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.885420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a842d79-13b0-41cb-9a4f-bdab434f901a admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.886092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef0001-1d1c-4c3f-9c43-92df4e9ef139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.886744 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef0001-1d1c-4c3f-9c43-92df4e9ef139 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.887023 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef0001-1d1c-4c3f-9c43-92df4e9ef139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.887293 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef0001-1d1c-4c3f-9c43-92df4e9ef139 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.887975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef0001-1d1c-4c3f-9c43-92df4e9ef139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.889878 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef0001-1d1c-4c3f-9c43-92df4e9ef139 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:49.918471 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 794/1577] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/users => generated 283 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:49.918471 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef0001-1d1c-4c3f-9c43-92df4e9ef139 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.922163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 784/1578] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:49.922163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a842d79-13b0-41cb-9a4f-bdab434f901a admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:49.926396 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef0001-1d1c-4c3f-9c43-92df4e9ef139 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.981732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a842d79-13b0-41cb-9a4f-bdab434f901a admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:49.985791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f0b25-9326-4612-8dd7-e3b2c9a3c1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.986120 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd23cf-cf6a-495b-be14-303aa7fc1e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:49.986414 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f0b25-9326-4612-8dd7-e3b2c9a3c1d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.986701 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f0b25-9326-4612-8dd7-e3b2c9a3c1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.986854 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd23cf-cf6a-495b-be14-303aa7fc1e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:49.987039 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f0b25-9326-4612-8dd7-e3b2c9a3c1d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.987166 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd23cf-cf6a-495b-be14-303aa7fc1e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:49.987370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd23cf-cf6a-495b-be14-303aa7fc1e60 admin admin] PATH_INFO: `/v3/projects/de98ffc315d4474c8b5d5370ba1fe97c/users/44655d95949b489088dbe48eeb8f38f7/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:49.987679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f0b25-9326-4612-8dd7-e3b2c9a3c1d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:49.989659 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f0b25-9326-4612-8dd7-e3b2c9a3c1d7 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.002697 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd23cf-cf6a-495b-be14-303aa7fc1e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de98ffc315d4474c8b5d5370ba1fe97c, user_id=44655d95949b489088dbe48eeb8f38f7, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.003735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: WARNING py.warnings [None req-9ccd23cf-cf6a-495b-be14-303aa7fc1e60 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 02:16:50.003735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: warnings.warn(msg) Aug 04 02:16:50.003735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: Aug 04 02:16:50.005744 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd23cf-cf6a-495b-be14-303aa7fc1e60 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.021834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 795/1579] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:16:49 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:50.022056 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 785/1580] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:16:49 2022] PUT /identity/v3/projects/de98ffc315d4474c8b5d5370ba1fe97c/users/44655d95949b489088dbe48eeb8f38f7/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:16:50.022156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f0b25-9326-4612-8dd7-e3b2c9a3c1d7 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.022238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd23cf-cf6a-495b-be14-303aa7fc1e60 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.082075 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd23cf-cf6a-495b-be14-303aa7fc1e60 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.083223 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f0b25-9326-4612-8dd7-e3b2c9a3c1d7 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.087190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463de6a-ecfd-4cbd-8ca8-8abe41f6ddf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.087801 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463de6a-ecfd-4cbd-8ca8-8abe41f6ddf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.088078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463de6a-ecfd-4cbd-8ca8-8abe41f6ddf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.088331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9059b-94ee-4892-ab7e-df783518a291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.088445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463de6a-ecfd-4cbd-8ca8-8abe41f6ddf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.088973 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9059b-94ee-4892-ab7e-df783518a291 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.089092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463de6a-ecfd-4cbd-8ca8-8abe41f6ddf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.089257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9059b-94ee-4892-ab7e-df783518a291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.089536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9059b-94ee-4892-ab7e-df783518a291 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.090192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9059b-94ee-4892-ab7e-df783518a291 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.090935 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463de6a-ecfd-4cbd-8ca8-8abe41f6ddf0 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.092093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9059b-94ee-4892-ab7e-df783518a291 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.117083 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 796/1581] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:16:50 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:50.117083 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9059b-94ee-4892-ab7e-df783518a291 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.118362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 786/1582] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:16:50 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:50.119179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463de6a-ecfd-4cbd-8ca8-8abe41f6ddf0 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.124453 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9059b-94ee-4892-ab7e-df783518a291 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.128257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:50.134899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:50.185251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c5085-378d-4e54-9982-ecfbc1c3382a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.185923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c5085-378d-4e54-9982-ecfbc1c3382a admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.186203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c5085-378d-4e54-9982-ecfbc1c3382a admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.186486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c5085-378d-4e54-9982-ecfbc1c3382a admin admin] PATH_INFO: `/v3/projects/c61d4ea2b52c497caeb298fe8677488e/users/033f13b2943945d3abc4ff852b68d698/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.189389 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463de6a-ecfd-4cbd-8ca8-8abe41f6ddf0 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.195039 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dab1b-d1c5-4cb8-ad6f-0b9eaa93770c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.195659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dab1b-d1c5-4cb8-ad6f-0b9eaa93770c admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.195962 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dab1b-d1c5-4cb8-ad6f-0b9eaa93770c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.196239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dab1b-d1c5-4cb8-ad6f-0b9eaa93770c admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.197040 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4dab1b-d1c5-4cb8-ad6f-0b9eaa93770c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.198650 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4dab1b-d1c5-4cb8-ad6f-0b9eaa93770c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.203665 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c5085-378d-4e54-9982-ecfbc1c3382a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c61d4ea2b52c497caeb298fe8677488e, user_id=033f13b2943945d3abc4ff852b68d698, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.207048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c5085-378d-4e54-9982-ecfbc1c3382a admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.213101 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 787/1583] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:50.213101 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dab1b-d1c5-4cb8-ad6f-0b9eaa93770c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.220984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dab1b-d1c5-4cb8-ad6f-0b9eaa93770c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.226341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 797/1584] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:16:50 2022] PUT /identity/v3/projects/c61d4ea2b52c497caeb298fe8677488e/users/033f13b2943945d3abc4ff852b68d698/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:16:50.227431 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c5085-378d-4e54-9982-ecfbc1c3382a admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.281235 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3753b-463d-48fd-ab31-0ea5aaa8e130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.281770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3753b-463d-48fd-ab31-0ea5aaa8e130 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.282055 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3753b-463d-48fd-ab31-0ea5aaa8e130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.282330 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3753b-463d-48fd-ab31-0ea5aaa8e130 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.283022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3753b-463d-48fd-ab31-0ea5aaa8e130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.284655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3753b-463d-48fd-ab31-0ea5aaa8e130 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.288909 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c5085-378d-4e54-9982-ecfbc1c3382a admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.293579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14623181-7164-446e-ac01-ccdaac881241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.294193 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14623181-7164-446e-ac01-ccdaac881241 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.294464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 788/1585] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:50.297461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14623181-7164-446e-ac01-ccdaac881241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.297461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14623181-7164-446e-ac01-ccdaac881241 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.297612 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3753b-463d-48fd-ab31-0ea5aaa8e130 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.297716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14623181-7164-446e-ac01-ccdaac881241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.297716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14623181-7164-446e-ac01-ccdaac881241 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.305233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3753b-463d-48fd-ab31-0ea5aaa8e130 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.310042 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba98494-a32d-47f0-a52a-3047af736bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.310758 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba98494-a32d-47f0-a52a-3047af736bec admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.311065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba98494-a32d-47f0-a52a-3047af736bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.311341 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba98494-a32d-47f0-a52a-3047af736bec admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.312090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba98494-a32d-47f0-a52a-3047af736bec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.314239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba98494-a32d-47f0-a52a-3047af736bec admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.320674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 798/1586] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:16:50 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:50.321922 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0640b-d9de-43bc-bac9-b1eae723324e None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.322210 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0640b-d9de-43bc-bac9-b1eae723324e None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.322679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0640b-d9de-43bc-bac9-b1eae723324e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.339194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 789/1587] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:16:50 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:50.340024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba98494-a32d-47f0-a52a-3047af736bec admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.343550 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-def0640b-d9de-43bc-bac9-b1eae723324e None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:50.344761 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-def0640b-d9de-43bc-bac9-b1eae723324e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:50.350250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba98494-a32d-47f0-a52a-3047af736bec admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.355624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 799/1588] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:50.355624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0640b-d9de-43bc-bac9-b1eae723324e None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.358491 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eba22e-ff24-42a0-b505-2441578de569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.359338 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eba22e-ff24-42a0-b505-2441578de569 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.359625 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eba22e-ff24-42a0-b505-2441578de569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.359900 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eba22e-ff24-42a0-b505-2441578de569 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.360606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eba22e-ff24-42a0-b505-2441578de569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.362244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08eba22e-ff24-42a0-b505-2441578de569 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.363061 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0640b-d9de-43bc-bac9-b1eae723324e None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.367648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3414289-cd20-4bd4-9127-a89cc181c39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.372996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3414289-cd20-4bd4-9127-a89cc181c39e admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.372996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3414289-cd20-4bd4-9127-a89cc181c39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.372996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3414289-cd20-4bd4-9127-a89cc181c39e admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.372996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3414289-cd20-4bd4-9127-a89cc181c39e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.372996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3414289-cd20-4bd4-9127-a89cc181c39e admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.373827 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 790/1589] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:50.377727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eba22e-ff24-42a0-b505-2441578de569 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.390778 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eba22e-ff24-42a0-b505-2441578de569 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.394864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 800/1590] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:16:50 2022] POST /identity/v3/users => generated 293 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:50.395947 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164743fb-4639-42f2-bfbc-4ac6c2032163 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.396230 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164743fb-4639-42f2-bfbc-4ac6c2032163 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.396505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164743fb-4639-42f2-bfbc-4ac6c2032163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.398241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9026e49-e1f2-4cb3-87d7-986868af81dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.398925 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9026e49-e1f2-4cb3-87d7-986868af81dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.399201 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9026e49-e1f2-4cb3-87d7-986868af81dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.399501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9026e49-e1f2-4cb3-87d7-986868af81dc admin admin] PATH_INFO: `/v3/projects/b447a897a70c421a8fede506b0f8afa1/users/9d4fea99de63408a8cf98f591c8a8a3e/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.414641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9026e49-e1f2-4cb3-87d7-986868af81dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b447a897a70c421a8fede506b0f8afa1, user_id=9d4fea99de63408a8cf98f591c8a8a3e, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.415411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-164743fb-4639-42f2-bfbc-4ac6c2032163 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:50.416651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-164743fb-4639-42f2-bfbc-4ac6c2032163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:50.417493 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9026e49-e1f2-4cb3-87d7-986868af81dc admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.426056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 801/1591] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:50.426989 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164743fb-4639-42f2-bfbc-4ac6c2032163 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.441114 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 791/1592] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:16:50 2022] PUT /identity/v3/projects/b447a897a70c421a8fede506b0f8afa1/users/9d4fea99de63408a8cf98f591c8a8a3e/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:16:50.441114 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9026e49-e1f2-4cb3-87d7-986868af81dc admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.441414 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164743fb-4639-42f2-bfbc-4ac6c2032163 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.492448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:50.508410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30beb21-704c-4b97-a2f5-a85ef7ca0f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.509469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30beb21-704c-4b97-a2f5-a85ef7ca0f32 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.509756 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30beb21-704c-4b97-a2f5-a85ef7ca0f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.510032 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30beb21-704c-4b97-a2f5-a85ef7ca0f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.510731 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30beb21-704c-4b97-a2f5-a85ef7ca0f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.512389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30beb21-704c-4b97-a2f5-a85ef7ca0f32 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.516296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:50.529033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 802/1593] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:50.530594 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30beb21-704c-4b97-a2f5-a85ef7ca0f32 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.533722 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9026e49-e1f2-4cb3-87d7-986868af81dc admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.538297 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85bcf1-04f8-45dd-b991-73510a6f244c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.538920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85bcf1-04f8-45dd-b991-73510a6f244c admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.539198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85bcf1-04f8-45dd-b991-73510a6f244c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.539471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad85bcf1-04f8-45dd-b991-73510a6f244c admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.540195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad85bcf1-04f8-45dd-b991-73510a6f244c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.541300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30beb21-704c-4b97-a2f5-a85ef7ca0f32 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.541833 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad85bcf1-04f8-45dd-b991-73510a6f244c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.545940 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6ebf1-0f7f-4969-b73f-d5d9148ae23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.546546 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6ebf1-0f7f-4969-b73f-d5d9148ae23b admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.546828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6ebf1-0f7f-4969-b73f-d5d9148ae23b admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.547108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6ebf1-0f7f-4969-b73f-d5d9148ae23b admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.547793 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6ebf1-0f7f-4969-b73f-d5d9148ae23b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.549576 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6ebf1-0f7f-4969-b73f-d5d9148ae23b admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.552115 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 792/1594] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:50.552952 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85bcf1-04f8-45dd-b991-73510a6f244c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.560964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 803/1595] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:50.562061 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6ebf1-0f7f-4969-b73f-d5d9148ae23b admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.572497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6ebf1-0f7f-4969-b73f-d5d9148ae23b admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.577212 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62a7c2-7b7a-455b-a435-b589c96b2ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.577806 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62a7c2-7b7a-455b-a435-b589c96b2ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.578104 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62a7c2-7b7a-455b-a435-b589c96b2ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.578369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf62a7c2-7b7a-455b-a435-b589c96b2ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.579048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62a7c2-7b7a-455b-a435-b589c96b2ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.580939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf62a7c2-7b7a-455b-a435-b589c96b2ed5 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.591845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 804/1596] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:50.592665 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62a7c2-7b7a-455b-a435-b589c96b2ed5 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.602148 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf62a7c2-7b7a-455b-a435-b589c96b2ed5 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.606679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdca685-6475-46e4-8503-21b09e9da20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.607274 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdca685-6475-46e4-8503-21b09e9da20a admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.607545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdca685-6475-46e4-8503-21b09e9da20a admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.607813 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdca685-6475-46e4-8503-21b09e9da20a admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.608488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdca685-6475-46e4-8503-21b09e9da20a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.610083 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdca685-6475-46e4-8503-21b09e9da20a admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.615553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad85bcf1-04f8-45dd-b991-73510a6f244c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.619919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c1a19-f4df-4af3-9651-83c76e740cb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.620565 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 805/1597] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:50.620968 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c1a19-f4df-4af3-9651-83c76e740cb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.621324 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c1a19-f4df-4af3-9651-83c76e740cb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.621431 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdca685-6475-46e4-8503-21b09e9da20a admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.621531 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c1a19-f4df-4af3-9651-83c76e740cb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.631441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c1a19-f4df-4af3-9651-83c76e740cb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.633325 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdca685-6475-46e4-8503-21b09e9da20a admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.633572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c1a19-f4df-4af3-9651-83c76e740cb1 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.636979 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0250072-96d0-4242-b50c-d6c3a6945c99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.637602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0250072-96d0-4242-b50c-d6c3a6945c99 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.637834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0250072-96d0-4242-b50c-d6c3a6945c99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.638109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0250072-96d0-4242-b50c-d6c3a6945c99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.643056 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 793/1598] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:16:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:16:50.644385 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ef87d-046c-4200-b0aa-8e1de1020c39 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.644691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ef87d-046c-4200-b0aa-8e1de1020c39 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.644993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002ef87d-046c-4200-b0aa-8e1de1020c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.651060 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0250072-96d0-4242-b50c-d6c3a6945c99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.652961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0250072-96d0-4242-b50c-d6c3a6945c99 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.664537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 806/1599] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:16:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:16:50.665732 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0250072-96d0-4242-b50c-d6c3a6945c99 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.675346 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0250072-96d0-4242-b50c-d6c3a6945c99 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.676720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-002ef87d-046c-4200-b0aa-8e1de1020c39 None None] MFA Rules not processed for user `9d4fea99de63408a8cf98f591c8a8a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:50.678076 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-002ef87d-046c-4200-b0aa-8e1de1020c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:50.681702 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695a564-0f72-44c0-b3ec-99e6b711f9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.682287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695a564-0f72-44c0-b3ec-99e6b711f9dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.682542 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695a564-0f72-44c0-b3ec-99e6b711f9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.682820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695a564-0f72-44c0-b3ec-99e6b711f9dc admin admin] PATH_INFO: `/v3/projects/280553ef422943e3b37e63b7adc9ded2/users/174d40665dff4994bf194842fde391ba/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.701678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695a564-0f72-44c0-b3ec-99e6b711f9dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=280553ef422943e3b37e63b7adc9ded2, user_id=174d40665dff4994bf194842fde391ba, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.704910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a695a564-0f72-44c0-b3ec-99e6b711f9dc admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.721818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 807/1600] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:16:50 2022] PUT /identity/v3/projects/280553ef422943e3b37e63b7adc9ded2/users/174d40665dff4994bf194842fde391ba/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:16:50.722809 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695a564-0f72-44c0-b3ec-99e6b711f9dc admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.751221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 794/1601] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:50.752155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ef87d-046c-4200-b0aa-8e1de1020c39 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.769745 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002ef87d-046c-4200-b0aa-8e1de1020c39 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.824000 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695a564-0f72-44c0-b3ec-99e6b711f9dc admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.831271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa584a-23cd-4c22-a658-8c026de4cf73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.831948 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa584a-23cd-4c22-a658-8c026de4cf73 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.832844 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa584a-23cd-4c22-a658-8c026de4cf73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.832844 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa584a-23cd-4c22-a658-8c026de4cf73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.833427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa584a-23cd-4c22-a658-8c026de4cf73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.835060 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa584a-23cd-4c22-a658-8c026de4cf73 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.850664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 808/1602] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:50.850664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa584a-23cd-4c22-a658-8c026de4cf73 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.861098 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa584a-23cd-4c22-a658-8c026de4cf73 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.864786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bbb9b-feee-4c78-b405-92b24b06c893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.865417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bbb9b-feee-4c78-b405-92b24b06c893 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.865695 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bbb9b-feee-4c78-b405-92b24b06c893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.865969 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bbb9b-feee-4c78-b405-92b24b06c893 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.866972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629cded-83a8-4140-8064-22b39c1cee78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.867245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150bbb9b-feee-4c78-b405-92b24b06c893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.867756 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629cded-83a8-4140-8064-22b39c1cee78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.868103 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629cded-83a8-4140-8064-22b39c1cee78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.868438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629cded-83a8-4140-8064-22b39c1cee78 admin admin] PATH_INFO: `/v3/projects/11284ae13d2f4c5ba47517f672968623/users/df5d3de1afa945c3b0873050eca06c0c/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.869867 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150bbb9b-feee-4c78-b405-92b24b06c893 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.882802 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 795/1603] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:50.883867 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bbb9b-feee-4c78-b405-92b24b06c893 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.889190 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629cded-83a8-4140-8064-22b39c1cee78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11284ae13d2f4c5ba47517f672968623, user_id=df5d3de1afa945c3b0873050eca06c0c, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.893137 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629cded-83a8-4140-8064-22b39c1cee78 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.897088 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bbb9b-feee-4c78-b405-92b24b06c893 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.903282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116b0dd-fc92-4af3-91ad-d1ae4e37b562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:50.903989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116b0dd-fc92-4af3-91ad-d1ae4e37b562 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.904273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116b0dd-fc92-4af3-91ad-d1ae4e37b562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.904586 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116b0dd-fc92-4af3-91ad-d1ae4e37b562 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.905362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8116b0dd-fc92-4af3-91ad-d1ae4e37b562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:50.907064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8116b0dd-fc92-4af3-91ad-d1ae4e37b562 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:50.910524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 809/1604] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:16:50 2022] PUT /identity/v3/projects/11284ae13d2f4c5ba47517f672968623/users/df5d3de1afa945c3b0873050eca06c0c/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:16:50.912080 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb0a4c-1347-4047-af14-60ae32d8155f None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:50.912495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb0a4c-1347-4047-af14-60ae32d8155f None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:50.912871 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bb0a4c-1347-4047-af14-60ae32d8155f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:50.930922 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 796/1605] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:50.931903 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116b0dd-fc92-4af3-91ad-d1ae4e37b562 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:50.938561 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:50.943202 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116b0dd-fc92-4af3-91ad-d1ae4e37b562 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:50.954729 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-18bb0a4c-1347-4047-af14-60ae32d8155f None None] MFA Rules not processed for user `174d40665dff4994bf194842fde391ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:50.956399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-18bb0a4c-1347-4047-af14-60ae32d8155f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:50.997335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:51.023131 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201a4b2-9f21-4f34-9018-d0a54bb00056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:51.023761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201a4b2-9f21-4f34-9018-d0a54bb00056 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:51.024042 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201a4b2-9f21-4f34-9018-d0a54bb00056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:51.024370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201a4b2-9f21-4f34-9018-d0a54bb00056 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:51.025075 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201a4b2-9f21-4f34-9018-d0a54bb00056 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:51.027009 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201a4b2-9f21-4f34-9018-d0a54bb00056 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:51.037293 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 810/1606] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:51.038882 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb0a4c-1347-4047-af14-60ae32d8155f None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:51.051847 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bb0a4c-1347-4047-af14-60ae32d8155f None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:51.058619 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184ceb6-a918-4878-8469-42e52d22723f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:51.059469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184ceb6-a918-4878-8469-42e52d22723f admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:51.059748 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184ceb6-a918-4878-8469-42e52d22723f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:51.060018 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184ceb6-a918-4878-8469-42e52d22723f admin admin] PATH_INFO: `/v3/projects/1c32d82cce6541718e46908eaf9b98e9/users/26e73db77d93496883b66572d6bf35f2/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:51.077857 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184ceb6-a918-4878-8469-42e52d22723f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c32d82cce6541718e46908eaf9b98e9, user_id=26e73db77d93496883b66572d6bf35f2, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:51.078488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 797/1607] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:16:50 2022] POST /identity/v3/projects => generated 363 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:51.080901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184ceb6-a918-4878-8469-42e52d22723f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:51.082018 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201a4b2-9f21-4f34-9018-d0a54bb00056 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:51.099077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 811/1608] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:16:51 2022] PUT /identity/v3/projects/1c32d82cce6541718e46908eaf9b98e9/users/26e73db77d93496883b66572d6bf35f2/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:16:51.099077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c60ebd-1f2a-4df2-a777-ee097c5d956c None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:51.099077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c60ebd-1f2a-4df2-a777-ee097c5d956c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:51.099077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c60ebd-1f2a-4df2-a777-ee097c5d956c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:51.116938 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-d2c60ebd-1f2a-4df2-a777-ee097c5d956c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:51.118162 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d2c60ebd-1f2a-4df2-a777-ee097c5d956c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:51.164000 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201a4b2-9f21-4f34-9018-d0a54bb00056 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:51.189753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 812/1609] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:51.190539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c60ebd-1f2a-4df2-a777-ee097c5d956c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:51.202558 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c60ebd-1f2a-4df2-a777-ee097c5d956c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:51.209479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6f773-d1e1-41a9-9833-986c73721c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:51.210563 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6f773-d1e1-41a9-9833-986c73721c1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:51.210890 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6f773-d1e1-41a9-9833-986c73721c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:51.211350 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6f773-d1e1-41a9-9833-986c73721c1f admin admin] PATH_INFO: `/v3/projects/2f003cc2b7104c369e1aa74d6e48a9fc/users/76703dc82dc644ef8d32f7e259ddea6f/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:51.233591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6f773-d1e1-41a9-9833-986c73721c1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f003cc2b7104c369e1aa74d6e48a9fc, user_id=76703dc82dc644ef8d32f7e259ddea6f, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:51.236425 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6f773-d1e1-41a9-9833-986c73721c1f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:51.242579 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae0a2d-fbb6-4956-91b5-a32fb7e45062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:51.243072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae0a2d-fbb6-4956-91b5-a32fb7e45062 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:51.243375 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae0a2d-fbb6-4956-91b5-a32fb7e45062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:51.243645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ae0a2d-fbb6-4956-91b5-a32fb7e45062 admin admin] PATH_INFO: `/v3/projects/1ccd522731d34716b2f806e4e51ba8e3/users/1328d755f6914570b51181fdecefdc7c/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:51.262992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 813/1610] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:16:51 2022] PUT /identity/v3/projects/2f003cc2b7104c369e1aa74d6e48a9fc/users/76703dc82dc644ef8d32f7e259ddea6f/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:16:51.262992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6f773-d1e1-41a9-9833-986c73721c1f admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:51.268990 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae0a2d-fbb6-4956-91b5-a32fb7e45062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ccd522731d34716b2f806e4e51ba8e3, user_id=1328d755f6914570b51181fdecefdc7c, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:51.272081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ae0a2d-fbb6-4956-91b5-a32fb7e45062 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:51.296070 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 798/1611] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:16:51 2022] PUT /identity/v3/projects/1ccd522731d34716b2f806e4e51ba8e3/users/1328d755f6914570b51181fdecefdc7c/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:16:51.301043 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae0a2d-fbb6-4956-91b5-a32fb7e45062 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:51.350325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:51.364356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6f773-d1e1-41a9-9833-986c73721c1f admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:51.422601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ae0a2d-fbb6-4956-91b5-a32fb7e45062 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:51.429393 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df210a22-4eb4-44bc-8d88-0f8fb3c46103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:51.430156 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df210a22-4eb4-44bc-8d88-0f8fb3c46103 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:51.430540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:51.430735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df210a22-4eb4-44bc-8d88-0f8fb3c46103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:51.431013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df210a22-4eb4-44bc-8d88-0f8fb3c46103 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:51.431739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df210a22-4eb4-44bc-8d88-0f8fb3c46103 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:51.434417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df210a22-4eb4-44bc-8d88-0f8fb3c46103 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:51.442253 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3872e-5834-4af2-a367-c5475c63ced0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:51.443242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3872e-5834-4af2-a367-c5475c63ced0 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:51.443722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3872e-5834-4af2-a367-c5475c63ced0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:51.444214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3872e-5834-4af2-a367-c5475c63ced0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:51.475175 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 799/1612] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:16:51 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:51.477453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4e092-7b51-4313-9e69-33d4ac3ed5c6 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:51.477739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4e092-7b51-4313-9e69-33d4ac3ed5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:51.478439 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4e092-7b51-4313-9e69-33d4ac3ed5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:51.513693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-7dd4e092-7b51-4313-9e69-33d4ac3ed5c6 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:51.515384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7dd4e092-7b51-4313-9e69-33d4ac3ed5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:51.524019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3872e-5834-4af2-a367-c5475c63ced0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:51.525904 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 800/1613] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:51.526068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3872e-5834-4af2-a367-c5475c63ced0 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:51.526683 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4e092-7b51-4313-9e69-33d4ac3ed5c6 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:51.539800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 814/1614] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:16:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:16:51.541861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04f6ed-76f7-4ab7-b1eb-2df42e45ba73 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:51.542248 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04f6ed-76f7-4ab7-b1eb-2df42e45ba73 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:51.542694 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04f6ed-76f7-4ab7-b1eb-2df42e45ba73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:51.570011 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-1c04f6ed-76f7-4ab7-b1eb-2df42e45ba73 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:51.571710 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1c04f6ed-76f7-4ab7-b1eb-2df42e45ba73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:51.582271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 815/1615] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:51.587244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b65ea-2f26-4dbf-9519-7eab3637ffef None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:51.587781 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b65ea-2f26-4dbf-9519-7eab3637ffef None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:51.588156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b65ea-2f26-4dbf-9519-7eab3637ffef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:51.601037 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4e092-7b51-4313-9e69-33d4ac3ed5c6 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:51.605630 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6231a-a171-4105-9864-a3166f8f4ec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:51.606378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6231a-a171-4105-9864-a3166f8f4ec6 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:51.606728 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6231a-a171-4105-9864-a3166f8f4ec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:51.607883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6231a-a171-4105-9864-a3166f8f4ec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:51.621235 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6231a-a171-4105-9864-a3166f8f4ec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:51.624377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6231a-a171-4105-9864-a3166f8f4ec6 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:51.627203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-cb3b65ea-2f26-4dbf-9519-7eab3637ffef None None] MFA Rules not processed for user `1328d755f6914570b51181fdecefdc7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:51.629256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-cb3b65ea-2f26-4dbf-9519-7eab3637ffef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:51.636204 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 801/1616] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:16:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:16:51.637398 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6231a-a171-4105-9864-a3166f8f4ec6 service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:51.671082 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6231a-a171-4105-9864-a3166f8f4ec6 service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:51.681805 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9596b-2863-48d2-92a0-de18fbafdd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:51.683242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9596b-2863-48d2-92a0-de18fbafdd8d admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:51.683711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9596b-2863-48d2-92a0-de18fbafdd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:51.684374 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9596b-2863-48d2-92a0-de18fbafdd8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:51.686908 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa9596b-2863-48d2-92a0-de18fbafdd8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:51.689455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa9596b-2863-48d2-92a0-de18fbafdd8d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:51.712059 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 802/1617] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:51.713678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9596b-2863-48d2-92a0-de18fbafdd8d admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:51.742837 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa9596b-2863-48d2-92a0-de18fbafdd8d admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:51.750755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 816/1618] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:51.751965 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b65ea-2f26-4dbf-9519-7eab3637ffef None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:51.766421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24912f4-2ae0-4831-99b0-10f8eaba4c0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:51.767248 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24912f4-2ae0-4831-99b0-10f8eaba4c0c service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:51.767686 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24912f4-2ae0-4831-99b0-10f8eaba4c0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:51.768421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24912f4-2ae0-4831-99b0-10f8eaba4c0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:51.784564 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b65ea-2f26-4dbf-9519-7eab3637ffef None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:51.792691 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51205904-af6f-421b-b385-bc86f8e78398 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:51.793326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51205904-af6f-421b-b385-bc86f8e78398 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:51.793605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51205904-af6f-421b-b385-bc86f8e78398 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:51.793877 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51205904-af6f-421b-b385-bc86f8e78398 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:51.808489 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24912f4-2ae0-4831-99b0-10f8eaba4c0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:51.809923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24912f4-2ae0-4831-99b0-10f8eaba4c0c service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:51.829238 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51205904-af6f-421b-b385-bc86f8e78398 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:51.830930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51205904-af6f-421b-b385-bc86f8e78398 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:51.841344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 803/1619] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:16:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:16:51.842490 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24912f4-2ae0-4831-99b0-10f8eaba4c0c service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:51.862976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24912f4-2ae0-4831-99b0-10f8eaba4c0c service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:51.865723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 817/1620] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:16:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:16:51.870834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a11e99-9129-4d3c-928e-59b0c42b494f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:51.872164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017023ae-818d-4593-92f1-3442fd6ae42e None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:51.872839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017023ae-818d-4593-92f1-3442fd6ae42e None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:51.873183 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017023ae-818d-4593-92f1-3442fd6ae42e None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:51.876666 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 818/1621] 10.222.0.45 () {54 vars in 962 bytes} [Thu Aug 4 02:16:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:16:51.885914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a11e99-9129-4d3c-928e-59b0c42b494f admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:51.886209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a11e99-9129-4d3c-928e-59b0c42b494f admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:51.886497 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a11e99-9129-4d3c-928e-59b0c42b494f admin admin] PATH_INFO: `/v3/projects/96e2c83a591f4384b4d83e7724120d81/users/a44d2730d60242d4b523c8cf0534ffd4/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:51.887071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c86431-db92-476c-9c3b-b2c4123da642 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:51.887357 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c86431-db92-476c-9c3b-b2c4123da642 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:51.887634 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c86431-db92-476c-9c3b-b2c4123da642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:51.909496 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a11e99-9129-4d3c-928e-59b0c42b494f admin admin] RBAC: Authorizing `identity:create_grant(project_id=96e2c83a591f4384b4d83e7724120d81, user_id=a44d2730d60242d4b523c8cf0534ffd4, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:51.912197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a11e99-9129-4d3c-928e-59b0c42b494f admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:51.940911 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 804/1622] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:16:51 2022] PUT /identity/v3/projects/96e2c83a591f4384b4d83e7724120d81/users/a44d2730d60242d4b523c8cf0534ffd4/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:16:51.941893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a11e99-9129-4d3c-928e-59b0c42b494f admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:52.007625 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-b3c86431-db92-476c-9c3b-b2c4123da642 None None] MFA Rules not processed for user `c571406d2a3f40dab035a918399d6f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:52.010264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b3c86431-db92-476c-9c3b-b2c4123da642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:52.035310 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:52.041463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a11e99-9129-4d3c-928e-59b0c42b494f admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:52.054816 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ffc39-293d-449a-ab18-f2776a303478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:52.055876 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ffc39-293d-449a-ab18-f2776a303478 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:52.056160 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ffc39-293d-449a-ab18-f2776a303478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:52.056432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ffc39-293d-449a-ab18-f2776a303478 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:52.057341 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ffc39-293d-449a-ab18-f2776a303478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:52.059201 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ffc39-293d-449a-ab18-f2776a303478 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:52.076848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 805/1623] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:16:51 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:16:52.078133 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ffc39-293d-449a-ab18-f2776a303478 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:52.093556 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ffc39-293d-449a-ab18-f2776a303478 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:52.103048 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bdf0eb-e7f4-4c59-9ed4-cae1595bb308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:52.103832 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdf0eb-e7f4-4c59-9ed4-cae1595bb308 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:52.104116 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdf0eb-e7f4-4c59-9ed4-cae1595bb308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:52.104392 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bdf0eb-e7f4-4c59-9ed4-cae1595bb308 admin admin] PATH_INFO: `/v3/projects/96e2c83a591f4384b4d83e7724120d81/users/a44d2730d60242d4b523c8cf0534ffd4/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:52.106761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdf0eb-e7f4-4c59-9ed4-cae1595bb308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96e2c83a591f4384b4d83e7724120d81, user_id=a44d2730d60242d4b523c8cf0534ffd4, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:52.110503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bdf0eb-e7f4-4c59-9ed4-cae1595bb308 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:52.139442 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:52.155477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 806/1624] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:16:52 2022] PUT /identity/v3/projects/96e2c83a591f4384b4d83e7724120d81/users/a44d2730d60242d4b523c8cf0534ffd4/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:16:52.158909 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aac56-87f6-4db0-883c-feef0de2cd55 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:52.159978 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aac56-87f6-4db0-883c-feef0de2cd55 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:52.160263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aac56-87f6-4db0-883c-feef0de2cd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:52.211564 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 819/1625] 10.222.0.45 () {58 vars in 1059 bytes} [Thu Aug 4 02:16:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:16:52.213728 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c86431-db92-476c-9c3b-b2c4123da642 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:52.228177 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-6b9aac56-87f6-4db0-883c-feef0de2cd55 None None] MFA Rules not processed for user `a44d2730d60242d4b523c8cf0534ffd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:52.230618 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-6b9aac56-87f6-4db0-883c-feef0de2cd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:52.240395 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c86431-db92-476c-9c3b-b2c4123da642 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:52.353183 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d87d61-a738-4f1a-88df-df95553b1b05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:52.354386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d87d61-a738-4f1a-88df-df95553b1b05 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:52.354386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d87d61-a738-4f1a-88df-df95553b1b05 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:52.354910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d87d61-a738-4f1a-88df-df95553b1b05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:52.451468 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 807/1626] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:16:52.465970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9aac56-87f6-4db0-883c-feef0de2cd55 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:52.530520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d87d61-a738-4f1a-88df-df95553b1b05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:52.535995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d87d61-a738-4f1a-88df-df95553b1b05 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:52.570039 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 820/1627] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:16:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:16:52.637286 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9aac56-87f6-4db0-883c-feef0de2cd55 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:52.657549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b1d1e-524a-452a-be6c-578d57f50369 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:52.660672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b1d1e-524a-452a-be6c-578d57f50369 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:52.660672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b1d1e-524a-452a-be6c-578d57f50369 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:52.660672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b1d1e-524a-452a-be6c-578d57f50369 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:52.700046 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b1d1e-524a-452a-be6c-578d57f50369 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:52.702802 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b1d1e-524a-452a-be6c-578d57f50369 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:52.722406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 808/1628] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:16:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:16:52.869775 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916aeddd-4de8-451a-a3dd-01eeb3f44951 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:52.870652 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916aeddd-4de8-451a-a3dd-01eeb3f44951 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:52.871111 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916aeddd-4de8-451a-a3dd-01eeb3f44951 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:52.873198 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 821/1629] 10.222.0.45 () {54 vars in 912 bytes} [Thu Aug 4 02:16:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:16:52.879160 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ea55d-f9d6-4758-8fc2-7cc31a65f4a1 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:52.880743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ea55d-f9d6-4758-8fc2-7cc31a65f4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:52.881204 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ea55d-f9d6-4758-8fc2-7cc31a65f4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:52.912423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-da4ea55d-f9d6-4758-8fc2-7cc31a65f4a1 None None] MFA Rules not processed for user `c571406d2a3f40dab035a918399d6f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:52.916867 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-da4ea55d-f9d6-4758-8fc2-7cc31a65f4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:52.938826 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 809/1630] 10.222.0.45 () {58 vars in 1009 bytes} [Thu Aug 4 02:16:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:52.946493 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916aeddd-4de8-451a-a3dd-01eeb3f44951 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:52.967280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916aeddd-4de8-451a-a3dd-01eeb3f44951 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:52.988704 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc8b92-5ce4-41ea-91f0-f7bf7d65d677 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:52.993918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc8b92-5ce4-41ea-91f0-f7bf7d65d677 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:52.994594 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc8b92-5ce4-41ea-91f0-f7bf7d65d677 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:52.995191 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc8b92-5ce4-41ea-91f0-f7bf7d65d677 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:53.027737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc8b92-5ce4-41ea-91f0-f7bf7d65d677 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:53.030953 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc8b92-5ce4-41ea-91f0-f7bf7d65d677 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:53.053303 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 822/1631] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:16:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:16:53.607727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ea55d-f9d6-4758-8fc2-7cc31a65f4a1 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:53.808990 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ea55d-f9d6-4758-8fc2-7cc31a65f4a1 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:53.827470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8dd35-dd78-4eb9-9600-85f7587ffed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:53.827470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8dd35-dd78-4eb9-9600-85f7587ffed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:53.827470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8dd35-dd78-4eb9-9600-85f7587ffed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:53.827470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8dd35-dd78-4eb9-9600-85f7587ffed7 admin admin] PATH_INFO: `/v3/users/9d4fea99de63408a8cf98f591c8a8a3e` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:53.831007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e8dd35-dd78-4eb9-9600-85f7587ffed7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d4fea99de63408a8cf98f591c8a8a3e)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:53.834209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: WARNING py.warnings [None req-a4e8dd35-dd78-4eb9-9600-85f7587ffed7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 02:16:53.834209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: warnings.warn(msg) Aug 04 02:16:53.834209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: Aug 04 02:16:53.842404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e8dd35-dd78-4eb9-9600-85f7587ffed7 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:53.848395 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:54.056546 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a4e8dd35-dd78-4eb9-9600-85f7587ffed7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d4fea99de63408a8cf98f591c8a8a3e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:54.058695 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a4e8dd35-dd78-4eb9-9600-85f7587ffed7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d4fea99de63408a8cf98f591c8a8a3e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:54.060535 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a4e8dd35-dd78-4eb9-9600-85f7587ffed7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d4fea99de63408a8cf98f591c8a8a3e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:54.121482 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a4e8dd35-dd78-4eb9-9600-85f7587ffed7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d4fea99de63408a8cf98f591c8a8a3e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:54.165943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a4e8dd35-dd78-4eb9-9600-85f7587ffed7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d4fea99de63408a8cf98f591c8a8a3e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:54.197336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 810/1632] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:16:53 2022] DELETE /identity/v3/users/9d4fea99de63408a8cf98f591c8a8a3e => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:16:54.202261 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace220b-b4da-4dba-9b91-ab90ebfc800c None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:54.202783 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace220b-b4da-4dba-9b91-ab90ebfc800c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:54.203032 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace220b-b4da-4dba-9b91-ab90ebfc800c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:54.239117 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-1ace220b-b4da-4dba-9b91-ab90ebfc800c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:54.240362 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1ace220b-b4da-4dba-9b91-ab90ebfc800c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:54.383678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70453b3e-dcc4-40e5-87f6-586d5f983690 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:54.383942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70453b3e-dcc4-40e5-87f6-586d5f983690 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:54.384189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70453b3e-dcc4-40e5-87f6-586d5f983690 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:54.386738 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 811/1633] 10.222.0.45 () {54 vars in 962 bytes} [Thu Aug 4 02:16:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:16:54.399737 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fceac-c6d1-4ede-b8bd-b57fe3ee67b2 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:54.400072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fceac-c6d1-4ede-b8bd-b57fe3ee67b2 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:54.400300 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12fceac-c6d1-4ede-b8bd-b57fe3ee67b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:54.402500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 823/1634] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:54.423125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace220b-b4da-4dba-9b91-ab90ebfc800c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:54.424660 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1ace220b-b4da-4dba-9b91-ab90ebfc800c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:54.436036 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-c12fceac-c6d1-4ede-b8bd-b57fe3ee67b2 None None] MFA Rules not processed for user `c571406d2a3f40dab035a918399d6f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:54.439779 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-c12fceac-c6d1-4ede-b8bd-b57fe3ee67b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:54.458343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:54.556805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace220b-b4da-4dba-9b91-ab90ebfc800c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:54.562376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1afab-92d5-4910-9149-39297cee33ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:54.563404 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1afab-92d5-4910-9149-39297cee33ea service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:54.563921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1afab-92d5-4910-9149-39297cee33ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:54.564545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1afab-92d5-4910-9149-39297cee33ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:54.590517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c1afab-92d5-4910-9149-39297cee33ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:54.592886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c1afab-92d5-4910-9149-39297cee33ea service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:54.606317 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 824/1635] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:16:54.619420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 812/1636] 10.222.0.45 () {58 vars in 1059 bytes} [Thu Aug 4 02:16:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:54.625040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622a1ea-1ac7-4767-8bbb-200bbe5112a6 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:54.625546 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622a1ea-1ac7-4767-8bbb-200bbe5112a6 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:54.626085 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622a1ea-1ac7-4767-8bbb-200bbe5112a6 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:54.627627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 825/1637] 10.222.0.45 () {54 vars in 912 bytes} [Thu Aug 4 02:16:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:16:54.634713 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315ca78-8010-4b75-a945-090ddf6a1dae None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:54.635122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315ca78-8010-4b75-a945-090ddf6a1dae None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:54.635466 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315ca78-8010-4b75-a945-090ddf6a1dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:54.682558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-6315ca78-8010-4b75-a945-090ddf6a1dae None None] MFA Rules not processed for user `c571406d2a3f40dab035a918399d6f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:54.683911 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-6315ca78-8010-4b75-a945-090ddf6a1dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:54.697416 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 813/1638] 10.222.0.45 () {58 vars in 1009 bytes} [Thu Aug 4 02:16:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:54.716862 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622a1ea-1ac7-4767-8bbb-200bbe5112a6 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:54.718888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-6622a1ea-1ac7-4767-8bbb-200bbe5112a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:54.813337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622a1ea-1ac7-4767-8bbb-200bbe5112a6 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:54.819697 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5860f-dc92-4ef9-ab0a-75ff70c94173 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:54.820861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5860f-dc92-4ef9-ab0a-75ff70c94173 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:54.821480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5860f-dc92-4ef9-ab0a-75ff70c94173 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:54.821851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5860f-dc92-4ef9-ab0a-75ff70c94173 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:54.850275 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c5860f-dc92-4ef9-ab0a-75ff70c94173 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:54.852749 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c5860f-dc92-4ef9-ab0a-75ff70c94173 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:54.871452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 826/1639] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:16:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:16:55.691944 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315ca78-8010-4b75-a945-090ddf6a1dae None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:55.694669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-6315ca78-8010-4b75-a945-090ddf6a1dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:55.717792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315ca78-8010-4b75-a945-090ddf6a1dae None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:55.725823 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d3069-aa22-44da-94e7-b2315e69ddaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:55.728486 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d3069-aa22-44da-94e7-b2315e69ddaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:55.729403 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d3069-aa22-44da-94e7-b2315e69ddaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:55.730596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d3069-aa22-44da-94e7-b2315e69ddaf admin admin] PATH_INFO: `/v3/projects/b447a897a70c421a8fede506b0f8afa1` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:55.734081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d3069-aa22-44da-94e7-b2315e69ddaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b447a897a70c421a8fede506b0f8afa1)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:55.737657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d3069-aa22-44da-94e7-b2315e69ddaf admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:55.865166 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d44d3069-aa22-44da-94e7-b2315e69ddaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b447a897a70c421a8fede506b0f8afa1', 'request_id': 'req-d44d3069-aa22-44da-94e7-b2315e69ddaf'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:55.878259 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d44d3069-aa22-44da-94e7-b2315e69ddaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b447a897a70c421a8fede506b0f8afa1', 'request_id': 'req-d44d3069-aa22-44da-94e7-b2315e69ddaf'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:55.884445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:55.888607 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d44d3069-aa22-44da-94e7-b2315e69ddaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b447a897a70c421a8fede506b0f8afa1', 'request_id': 'req-d44d3069-aa22-44da-94e7-b2315e69ddaf'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:55.908421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d44d3069-aa22-44da-94e7-b2315e69ddaf admin admin] The token cache is being invalidate because project b447a897a70c421a8fede506b0f8afa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:16:55.909317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d44d3069-aa22-44da-94e7-b2315e69ddaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:55.910969 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d44d3069-aa22-44da-94e7-b2315e69ddaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:55.916004 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 814/1640] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:16:55 2022] DELETE /identity/v3/projects/b447a897a70c421a8fede506b0f8afa1 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:16:55.916465 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5860f-dc92-4ef9-ab0a-75ff70c94173 service cinder] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:55.930237 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c5860f-dc92-4ef9-ab0a-75ff70c94173 service cinder] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:56.062170 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36225787-980a-4f9c-8fe2-49e703570fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:56.062815 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36225787-980a-4f9c-8fe2-49e703570fed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:56.063101 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36225787-980a-4f9c-8fe2-49e703570fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:56.063388 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36225787-980a-4f9c-8fe2-49e703570fed admin admin] PATH_INFO: `/v3/users/a44d2730d60242d4b523c8cf0534ffd4` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:56.065291 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36225787-980a-4f9c-8fe2-49e703570fed admin admin] RBAC: Authorizing `identity:delete_user(user_id=a44d2730d60242d4b523c8cf0534ffd4)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:56.067309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36225787-980a-4f9c-8fe2-49e703570fed admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:56.185828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-36225787-980a-4f9c-8fe2-49e703570fed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a44d2730d60242d4b523c8cf0534ffd4'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:56.196691 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-36225787-980a-4f9c-8fe2-49e703570fed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a44d2730d60242d4b523c8cf0534ffd4'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:56.197546 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-36225787-980a-4f9c-8fe2-49e703570fed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a44d2730d60242d4b523c8cf0534ffd4'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:56.201082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:16:56.222222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-36225787-980a-4f9c-8fe2-49e703570fed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a44d2730d60242d4b523c8cf0534ffd4'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:56.224109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-36225787-980a-4f9c-8fe2-49e703570fed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a44d2730d60242d4b523c8cf0534ffd4'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:56.248157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 827/1641] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:16:55 2022] DELETE /identity/v3/users/a44d2730d60242d4b523c8cf0534ffd4 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:16:59.059196 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d3069-aa22-44da-94e7-b2315e69ddaf admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:59.060727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d44d3069-aa22-44da-94e7-b2315e69ddaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:59.097139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d3069-aa22-44da-94e7-b2315e69ddaf admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:59.232050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ef372e-ad2b-46a0-aa1d-c47e3b79fc69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:59.232292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef372e-ad2b-46a0-aa1d-c47e3b79fc69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:59.232516 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef372e-ad2b-46a0-aa1d-c47e3b79fc69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:59.232813 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ef372e-ad2b-46a0-aa1d-c47e3b79fc69 admin admin] PATH_INFO: `/v3/projects/96e2c83a591f4384b4d83e7724120d81` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:59.235470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef372e-ad2b-46a0-aa1d-c47e3b79fc69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96e2c83a591f4384b4d83e7724120d81)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:59.239975 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ef372e-ad2b-46a0-aa1d-c47e3b79fc69 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:59.314588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-26ef372e-ad2b-46a0-aa1d-c47e3b79fc69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96e2c83a591f4384b4d83e7724120d81', 'request_id': 'req-26ef372e-ad2b-46a0-aa1d-c47e3b79fc69'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:59.326032 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-26ef372e-ad2b-46a0-aa1d-c47e3b79fc69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96e2c83a591f4384b4d83e7724120d81', 'request_id': 'req-26ef372e-ad2b-46a0-aa1d-c47e3b79fc69'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:59.335979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-26ef372e-ad2b-46a0-aa1d-c47e3b79fc69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96e2c83a591f4384b4d83e7724120d81', 'request_id': 'req-26ef372e-ad2b-46a0-aa1d-c47e3b79fc69'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:59.356079 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-26ef372e-ad2b-46a0-aa1d-c47e3b79fc69 admin admin] The token cache is being invalidate because project 96e2c83a591f4384b4d83e7724120d81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:16:59.357156 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-26ef372e-ad2b-46a0-aa1d-c47e3b79fc69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:59.358343 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-26ef372e-ad2b-46a0-aa1d-c47e3b79fc69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:16:59.364404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 815/1642] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:16:59 2022] DELETE /identity/v3/projects/96e2c83a591f4384b4d83e7724120d81 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:16:59.365849 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf8faf-57fa-4cab-a9e0-95f39655c60f None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:59.366450 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf8faf-57fa-4cab-a9e0-95f39655c60f None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:59.366450 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf8faf-57fa-4cab-a9e0-95f39655c60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:59.400762 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-aabf8faf-57fa-4cab-a9e0-95f39655c60f None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:59.402448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-aabf8faf-57fa-4cab-a9e0-95f39655c60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:59.588697 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 828/1643] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:59.597179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13f1ef-d02a-437d-8193-4ef370dc38bc None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:59.597179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13f1ef-d02a-437d-8193-4ef370dc38bc None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:59.597465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13f1ef-d02a-437d-8193-4ef370dc38bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:59.657649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-7d13f1ef-d02a-437d-8193-4ef370dc38bc None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:16:59.659595 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7d13f1ef-d02a-437d-8193-4ef370dc38bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:16:59.691758 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 816/1644] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:16:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:16:59.691952 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf8faf-57fa-4cab-a9e0-95f39655c60f None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:59.711390 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf8faf-57fa-4cab-a9e0-95f39655c60f None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:59.726773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283bf72-f619-40df-bac8-096428c334a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:59.727696 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283bf72-f619-40df-bac8-096428c334a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:59.729155 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283bf72-f619-40df-bac8-096428c334a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:59.730112 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283bf72-f619-40df-bac8-096428c334a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:59.732363 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283bf72-f619-40df-bac8-096428c334a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:16:59.732778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283bf72-f619-40df-bac8-096428c334a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:59.737060 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283bf72-f619-40df-bac8-096428c334a8 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:59.806490 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13f1ef-d02a-437d-8193-4ef370dc38bc None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:59.810064 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 829/1645] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:16:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:16:59.826860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13f1ef-d02a-437d-8193-4ef370dc38bc None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:16:59.849383 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec06d1c-29ca-42a3-a17a-691ec6c58e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:16:59.850035 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec06d1c-29ca-42a3-a17a-691ec6c58e13 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:16:59.850316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec06d1c-29ca-42a3-a17a-691ec6c58e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:16:59.851231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec06d1c-29ca-42a3-a17a-691ec6c58e13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:16:59.851913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec06d1c-29ca-42a3-a17a-691ec6c58e13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:16:59.853898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec06d1c-29ca-42a3-a17a-691ec6c58e13 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:16:59.952562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283bf72-f619-40df-bac8-096428c334a8 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:16:59.956771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 817/1646] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:16:59 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:16:59.989629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283bf72-f619-40df-bac8-096428c334a8 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:00.255723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389e930-5ad8-4e59-90c1-eee9d17289fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:00.257006 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389e930-5ad8-4e59-90c1-eee9d17289fe admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:00.257758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389e930-5ad8-4e59-90c1-eee9d17289fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:00.257758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2389e930-5ad8-4e59-90c1-eee9d17289fe admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:00.258466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389e930-5ad8-4e59-90c1-eee9d17289fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:00.260521 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2389e930-5ad8-4e59-90c1-eee9d17289fe admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:00.315813 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 830/1647] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:16:59 2022] POST /identity/v3/users => generated 290 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:00.318428 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec06d1c-29ca-42a3-a17a-691ec6c58e13 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:00.343256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec06d1c-29ca-42a3-a17a-691ec6c58e13 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:00.354840 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d196965-22f5-4b7a-80fd-155e0320d38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:00.355695 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d196965-22f5-4b7a-80fd-155e0320d38f admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:00.356151 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d196965-22f5-4b7a-80fd-155e0320d38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:00.356588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d196965-22f5-4b7a-80fd-155e0320d38f admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:00.357506 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d196965-22f5-4b7a-80fd-155e0320d38f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:00.359424 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d196965-22f5-4b7a-80fd-155e0320d38f admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:00.374377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 818/1648] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:00.378786 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389e930-5ad8-4e59-90c1-eee9d17289fe admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:00.394311 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2389e930-5ad8-4e59-90c1-eee9d17289fe admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:00.417231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a05b55-4f40-462e-b9ed-8dfa18d14b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:00.417231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a05b55-4f40-462e-b9ed-8dfa18d14b94 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:00.417231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a05b55-4f40-462e-b9ed-8dfa18d14b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:00.417231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a05b55-4f40-462e-b9ed-8dfa18d14b94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:00.417672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a05b55-4f40-462e-b9ed-8dfa18d14b94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:00.418801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a05b55-4f40-462e-b9ed-8dfa18d14b94 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:00.465586 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 831/1649] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:00 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:00.470374 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d196965-22f5-4b7a-80fd-155e0320d38f admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:00.498345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d196965-22f5-4b7a-80fd-155e0320d38f admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:00.509855 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bcfc0-04e5-4d64-b1b7-b49b25db7b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:00.510567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bcfc0-04e5-4d64-b1b7-b49b25db7b1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:00.510851 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bcfc0-04e5-4d64-b1b7-b49b25db7b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:00.511160 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bcfc0-04e5-4d64-b1b7-b49b25db7b1b admin admin] PATH_INFO: `/v3/projects/babea4c173ce42f49f43d8b09468b1e0/users/953e22189f6347c486717da01015f6f0/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:00.570097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bcfc0-04e5-4d64-b1b7-b49b25db7b1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=babea4c173ce42f49f43d8b09468b1e0, user_id=953e22189f6347c486717da01015f6f0, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:00.573951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bcfc0-04e5-4d64-b1b7-b49b25db7b1b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:00.605430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 819/1650] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:00 2022] PUT /identity/v3/projects/babea4c173ce42f49f43d8b09468b1e0/users/953e22189f6347c486717da01015f6f0/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:00.622837 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222beb9a-b14c-4092-8070-22f23adae34c None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:00.623561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222beb9a-b14c-4092-8070-22f23adae34c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:00.624043 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222beb9a-b14c-4092-8070-22f23adae34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:00.693154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-222beb9a-b14c-4092-8070-22f23adae34c None None] MFA Rules not processed for user `953e22189f6347c486717da01015f6f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:00.693154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-222beb9a-b14c-4092-8070-22f23adae34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:00.778888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:00.873973 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 832/1651] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:00.878740 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bcfc0-04e5-4d64-b1b7-b49b25db7b1b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:00.909855 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:01.060245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bcfc0-04e5-4d64-b1b7-b49b25db7b1b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:01.073717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c679bb5-eca1-49bd-8706-0ea97bbf9b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:01.078411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c679bb5-eca1-49bd-8706-0ea97bbf9b05 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:01.079094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c679bb5-eca1-49bd-8706-0ea97bbf9b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:01.080797 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c679bb5-eca1-49bd-8706-0ea97bbf9b05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:01.082806 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c679bb5-eca1-49bd-8706-0ea97bbf9b05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:01.087020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c679bb5-eca1-49bd-8706-0ea97bbf9b05 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:01.158704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 820/1652] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:00 2022] POST /identity/v3/projects => generated 381 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:01.164255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222beb9a-b14c-4092-8070-22f23adae34c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:01.188576 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222beb9a-b14c-4092-8070-22f23adae34c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:01.452137 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c12e9-3947-4eaa-84b8-1e4fa52010e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:01.453646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c12e9-3947-4eaa-84b8-1e4fa52010e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:01.454073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c12e9-3947-4eaa-84b8-1e4fa52010e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:01.454491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c12e9-3947-4eaa-84b8-1e4fa52010e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:01.455311 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c12e9-3947-4eaa-84b8-1e4fa52010e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:01.458510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c12e9-3947-4eaa-84b8-1e4fa52010e9 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:01.496416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 833/1653] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:01 2022] POST /identity/v3/users => generated 297 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:01.499592 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c679bb5-eca1-49bd-8706-0ea97bbf9b05 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:01.520058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c679bb5-eca1-49bd-8706-0ea97bbf9b05 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:01.538672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f0b01-163b-4564-ba85-8422e62b4405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:01.539258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f0b01-163b-4564-ba85-8422e62b4405 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:01.539539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f0b01-163b-4564-ba85-8422e62b4405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:01.539819 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9f0b01-163b-4564-ba85-8422e62b4405 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:01.545923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f0b01-163b-4564-ba85-8422e62b4405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:01.554197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9f0b01-163b-4564-ba85-8422e62b4405 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:01.594663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 821/1654] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:01.598597 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c12e9-3947-4eaa-84b8-1e4fa52010e9 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:01.623956 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c12e9-3947-4eaa-84b8-1e4fa52010e9 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:01.641015 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819fb87-497d-4211-8bf5-4ec9def13530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:01.641712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819fb87-497d-4211-8bf5-4ec9def13530 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:01.641993 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819fb87-497d-4211-8bf5-4ec9def13530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:01.642266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819fb87-497d-4211-8bf5-4ec9def13530 admin admin] PATH_INFO: `/v3/projects/d506cb539df146128fe763e275f29d03/users/9350726e88784a96905a89b7762d34a5/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:01.702387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819fb87-497d-4211-8bf5-4ec9def13530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d506cb539df146128fe763e275f29d03, user_id=9350726e88784a96905a89b7762d34a5, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:01.705612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d819fb87-497d-4211-8bf5-4ec9def13530 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:01.731055 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 834/1655] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:01 2022] PUT /identity/v3/projects/d506cb539df146128fe763e275f29d03/users/9350726e88784a96905a89b7762d34a5/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:01.733358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f0b01-163b-4564-ba85-8422e62b4405 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:01.779759 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9f0b01-163b-4564-ba85-8422e62b4405 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:02.014593 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbaf76b-98f2-428d-bdf7-922d55042931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:02.024512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbaf76b-98f2-428d-bdf7-922d55042931 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:02.024811 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbaf76b-98f2-428d-bdf7-922d55042931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:02.026331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbaf76b-98f2-428d-bdf7-922d55042931 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:02.026993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbaf76b-98f2-428d-bdf7-922d55042931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:02.032532 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbaf76b-98f2-428d-bdf7-922d55042931 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:02.063084 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 822/1656] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:02.064728 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819fb87-497d-4211-8bf5-4ec9def13530 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:02.086307 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819fb87-497d-4211-8bf5-4ec9def13530 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:02.092270 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43775c-8178-4d3d-80ef-23ff02806cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:02.093022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43775c-8178-4d3d-80ef-23ff02806cf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:02.093301 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43775c-8178-4d3d-80ef-23ff02806cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:02.093745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43775c-8178-4d3d-80ef-23ff02806cf8 admin admin] PATH_INFO: `/v3/projects/d506cb539df146128fe763e275f29d03/users/9350726e88784a96905a89b7762d34a5/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:02.096100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43775c-8178-4d3d-80ef-23ff02806cf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d506cb539df146128fe763e275f29d03, user_id=9350726e88784a96905a89b7762d34a5, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:02.100503 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43775c-8178-4d3d-80ef-23ff02806cf8 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:02.136223 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 835/1657] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:02 2022] PUT /identity/v3/projects/d506cb539df146128fe763e275f29d03/users/9350726e88784a96905a89b7762d34a5/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:02.144358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439336b-6d19-4945-af1e-93825aed1e10 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:02.144499 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439336b-6d19-4945-af1e-93825aed1e10 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:02.146169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2439336b-6d19-4945-af1e-93825aed1e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:02.215259 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-2439336b-6d19-4945-af1e-93825aed1e10 None None] MFA Rules not processed for user `9350726e88784a96905a89b7762d34a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:02.216875 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2439336b-6d19-4945-af1e-93825aed1e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:02.323610 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bd841-8036-4724-89b0-55a61940d49f None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:02.323610 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bd841-8036-4724-89b0-55a61940d49f None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:02.323610 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bd841-8036-4724-89b0-55a61940d49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:02.402459 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 823/1658] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:02.407295 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-ec0bd841-8036-4724-89b0-55a61940d49f None None] MFA Rules not processed for user `033f13b2943945d3abc4ff852b68d698`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:02.431926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439336b-6d19-4945-af1e-93825aed1e10 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:02.433668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2439336b-6d19-4945-af1e-93825aed1e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:02.441838 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-ec0bd841-8036-4724-89b0-55a61940d49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:02.476952 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:02.584340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2439336b-6d19-4945-af1e-93825aed1e10 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:02.602213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d107ef-a547-4f7d-8255-eb46c713c516 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:02.602871 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d107ef-a547-4f7d-8255-eb46c713c516 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:02.603171 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d107ef-a547-4f7d-8255-eb46c713c516 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:02.603434 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d107ef-a547-4f7d-8255-eb46c713c516 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:02.604997 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 836/1659] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:02 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:02.618015 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bf634-09a3-49b6-8684-c097c7f57607 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:02.626323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bf634-09a3-49b6-8684-c097c7f57607 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:02.626751 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299bf634-09a3-49b6-8684-c097c7f57607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:02.630334 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d107ef-a547-4f7d-8255-eb46c713c516 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:02.631423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d107ef-a547-4f7d-8255-eb46c713c516 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:02.665532 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 824/1660] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:02.666631 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d107ef-a547-4f7d-8255-eb46c713c516 service cinder] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:02.668182 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e7d107ef-a547-4f7d-8255-eb46c713c516 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:02.682765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-299bf634-09a3-49b6-8684-c097c7f57607 None None] MFA Rules not processed for user `033f13b2943945d3abc4ff852b68d698`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:02.686818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-299bf634-09a3-49b6-8684-c097c7f57607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:02.708609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 837/1661] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:02 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:02.723971 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bf634-09a3-49b6-8684-c097c7f57607 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:02.727657 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-299bf634-09a3-49b6-8684-c097c7f57607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:02.743490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:02.747581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d107ef-a547-4f7d-8255-eb46c713c516 service cinder] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:02.757820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c21cd1-21e9-4401-8eed-ea4f6054f544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:02.758738 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c21cd1-21e9-4401-8eed-ea4f6054f544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:02.759314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c21cd1-21e9-4401-8eed-ea4f6054f544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:02.759765 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c21cd1-21e9-4401-8eed-ea4f6054f544 admin admin] PATH_INFO: `/v3/users/1328d755f6914570b51181fdecefdc7c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:02.761236 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c21cd1-21e9-4401-8eed-ea4f6054f544 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1328d755f6914570b51181fdecefdc7c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:02.763510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c21cd1-21e9-4401-8eed-ea4f6054f544 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:02.829978 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-75c21cd1-21e9-4401-8eed-ea4f6054f544 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1328d755f6914570b51181fdecefdc7c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:02.831103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-75c21cd1-21e9-4401-8eed-ea4f6054f544 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1328d755f6914570b51181fdecefdc7c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:02.844145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-75c21cd1-21e9-4401-8eed-ea4f6054f544 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1328d755f6914570b51181fdecefdc7c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:02.866545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299bf634-09a3-49b6-8684-c097c7f57607 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:02.880936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-75c21cd1-21e9-4401-8eed-ea4f6054f544 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1328d755f6914570b51181fdecefdc7c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:02.890047 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fc78b-9b84-4872-bd38-82c53c2f05b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:02.891022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fc78b-9b84-4872-bd38-82c53c2f05b5 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:02.892537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fc78b-9b84-4872-bd38-82c53c2f05b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:02.892866 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-75c21cd1-21e9-4401-8eed-ea4f6054f544 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1328d755f6914570b51181fdecefdc7c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:02.893587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2fc78b-9b84-4872-bd38-82c53c2f05b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:02.900517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-bc2fc78b-9b84-4872-bd38-82c53c2f05b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:02.928693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 825/1662] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:02 2022] DELETE /identity/v3/users/1328d755f6914570b51181fdecefdc7c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:02.933555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d263bad-9584-465e-98c2-d741f4bcd526 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:02.934104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d263bad-9584-465e-98c2-d741f4bcd526 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:02.934986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d263bad-9584-465e-98c2-d741f4bcd526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:02.940386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fc78b-9b84-4872-bd38-82c53c2f05b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:02.942219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2fc78b-9b84-4872-bd38-82c53c2f05b5 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:02.957930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 838/1663] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:02 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:17:02.984119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-1d263bad-9584-465e-98c2-d741f4bcd526 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:02.985399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-1d263bad-9584-465e-98c2-d741f4bcd526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:03.070509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:03.158845 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 826/1664] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:03.384161 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fc78b-9b84-4872-bd38-82c53c2f05b5 service nova] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:03.386680 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-bc2fc78b-9b84-4872-bd38-82c53c2f05b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:03.413332 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2fc78b-9b84-4872-bd38-82c53c2f05b5 service nova] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:03.540749 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d6e3d-4e00-4f30-b8c4-1f50ed79cd16 tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9350726e88784a96905a89b7762d34a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd506cb539df146128fe763e275f29d03', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:03.541995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d6e3d-4e00-4f30-b8c4-1f50ed79cd16 tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:03.542527 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d6e3d-4e00-4f30-b8c4-1f50ed79cd16 tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:03.550769 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d6e3d-4e00-4f30-b8c4-1f50ed79cd16 tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:03.551872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d6e3d-4e00-4f30-b8c4-1f50ed79cd16 tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:17:03.552379 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d6e3d-4e00-4f30-b8c4-1f50ed79cd16 tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:03.555185 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d6e3d-4e00-4f30-b8c4-1f50ed79cd16 tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:03.589223 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d263bad-9584-465e-98c2-d741f4bcd526 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:03.591045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 839/1665] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:17:03.591907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-1d263bad-9584-465e-98c2-d741f4bcd526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:03.595014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d6e3d-4e00-4f30-b8c4-1f50ed79cd16 tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:03.612859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d6e3d-4e00-4f30-b8c4-1f50ed79cd16 tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:03.625455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd24b5-d9ef-4d4a-99e3-22fc68cf5d8f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9350726e88784a96905a89b7762d34a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd506cb539df146128fe763e275f29d03', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:03.625932 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd24b5-d9ef-4d4a-99e3-22fc68cf5d8f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:03.626223 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd24b5-d9ef-4d4a-99e3-22fc68cf5d8f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:03.626507 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacd24b5-d9ef-4d4a-99e3-22fc68cf5d8f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:03.627191 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd24b5-d9ef-4d4a-99e3-22fc68cf5d8f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:03.629173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacd24b5-d9ef-4d4a-99e3-22fc68cf5d8f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:03.710387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 840/1666] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:03 2022] POST /identity/v3/projects => generated 355 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:17:03.714827 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd24b5-d9ef-4d4a-99e3-22fc68cf5d8f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:03.734998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d263bad-9584-465e-98c2-d741f4bcd526 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:03.848420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f90e6-60e1-41a3-8ee3-61e172077cc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:03.849226 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f90e6-60e1-41a3-8ee3-61e172077cc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:03.849600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f90e6-60e1-41a3-8ee3-61e172077cc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:03.849771 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacd24b5-d9ef-4d4a-99e3-22fc68cf5d8f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:03.851508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f90e6-60e1-41a3-8ee3-61e172077cc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:03.856594 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fde48-8a57-4289-923c-61fc9ef73a2e tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9350726e88784a96905a89b7762d34a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd506cb539df146128fe763e275f29d03', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:03.859799 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fde48-8a57-4289-923c-61fc9ef73a2e tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:03.860252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fde48-8a57-4289-923c-61fc9ef73a2e tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:03.860948 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fde48-8a57-4289-923c-61fc9ef73a2e tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:03.862074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fde48-8a57-4289-923c-61fc9ef73a2e tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:17:03.862532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fde48-8a57-4289-923c-61fc9ef73a2e tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:03.864334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fde48-8a57-4289-923c-61fc9ef73a2e tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:03.877328 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:03.891025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 841/1667] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:17:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:17:03.924426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f90e6-60e1-41a3-8ee3-61e172077cc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:03.926082 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f90e6-60e1-41a3-8ee3-61e172077cc2 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:03.938254 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 827/1668] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:03.958800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fde48-8a57-4289-923c-61fc9ef73a2e tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:03.975292 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fde48-8a57-4289-923c-61fc9ef73a2e tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:03.981975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737d89a-9ed7-42e7-9530-53e5255aa02f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9350726e88784a96905a89b7762d34a5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd506cb539df146128fe763e275f29d03', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:03.983315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737d89a-9ed7-42e7-9530-53e5255aa02f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:03.984066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737d89a-9ed7-42e7-9530-53e5255aa02f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:03.984510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737d89a-9ed7-42e7-9530-53e5255aa02f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] PATH_INFO: `/v3/projects/8f414b771f1e4b79bd4a900365895970` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:03.986060 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737d89a-9ed7-42e7-9530-53e5255aa02f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] RBAC: Authorizing `identity:delete_project(project_id=8f414b771f1e4b79bd4a900365895970)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:03.987337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: WARNING py.warnings [None req-3737d89a-9ed7-42e7-9530-53e5255aa02f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 04 02:17:03.987337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: warnings.warn(msg) Aug 04 02:17:03.987337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: Aug 04 02:17:03.988905 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3737d89a-9ed7-42e7-9530-53e5255aa02f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:03.992485 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65113b97-e176-4bc3-b96e-0c5721af6cf9 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:03.993574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65113b97-e176-4bc3-b96e-0c5721af6cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:03.993948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65113b97-e176-4bc3-b96e-0c5721af6cf9 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:03.995767 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 828/1669] 10.222.0.45 () {54 vars in 912 bytes} [Thu Aug 4 02:17:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:17:04.010378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855d178-a677-4ef8-84b2-cf78850c176f None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:04.010378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855d178-a677-4ef8-84b2-cf78850c176f None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:04.010378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855d178-a677-4ef8-84b2-cf78850c176f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:04.103067 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-d855d178-a677-4ef8-84b2-cf78850c176f None None] MFA Rules not processed for user `6c2da427acb24c8cb6d667c6347858c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:04.104483 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d855d178-a677-4ef8-84b2-cf78850c176f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:04.117296 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3737d89a-9ed7-42e7-9530-53e5255aa02f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f414b771f1e4b79bd4a900365895970', 'request_id': 'req-3737d89a-9ed7-42e7-9530-53e5255aa02f'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:04.134779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3737d89a-9ed7-42e7-9530-53e5255aa02f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f414b771f1e4b79bd4a900365895970', 'request_id': 'req-3737d89a-9ed7-42e7-9530-53e5255aa02f'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:04.150491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3737d89a-9ed7-42e7-9530-53e5255aa02f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f414b771f1e4b79bd4a900365895970', 'request_id': 'req-3737d89a-9ed7-42e7-9530-53e5255aa02f'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:04.166534 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:04.199994 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3737d89a-9ed7-42e7-9530-53e5255aa02f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] The token cache is being invalidate because project 8f414b771f1e4b79bd4a900365895970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:04.200287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3737d89a-9ed7-42e7-9530-53e5255aa02f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:04.201331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3737d89a-9ed7-42e7-9530-53e5255aa02f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:04.208203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 842/1670] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:03 2022] DELETE /identity/v3/projects/8f414b771f1e4b79bd4a900365895970 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:04.259533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 829/1671] 10.222.0.45 () {58 vars in 1009 bytes} [Thu Aug 4 02:17:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:04.343470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737d89a-9ed7-42e7-9530-53e5255aa02f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:04.345420 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-3737d89a-9ed7-42e7-9530-53e5255aa02f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:04.387123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737d89a-9ed7-42e7-9530-53e5255aa02f tempest-VolumeQuotaClassesTest-1635552503 tempest-VolumeQuotaClassesTest-1635552503-project-admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:04.412015 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3cf3dc-c73c-4ba5-8b68-c07c1c3e1247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:04.419344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cf3dc-c73c-4ba5-8b68-c07c1c3e1247 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:04.419722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cf3dc-c73c-4ba5-8b68-c07c1c3e1247 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:04.420174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3cf3dc-c73c-4ba5-8b68-c07c1c3e1247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:04.449963 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3cf3dc-c73c-4ba5-8b68-c07c1c3e1247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:04.456671 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3cf3dc-c73c-4ba5-8b68-c07c1c3e1247 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:04.499934 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 843/1672] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:04.806073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855d178-a677-4ef8-84b2-cf78850c176f None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:04.807709 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d855d178-a677-4ef8-84b2-cf78850c176f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:05.053249 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855d178-a677-4ef8-84b2-cf78850c176f None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:05.067486 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5edc906-1855-4bfa-8262-3f3dae458070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:05.070929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edc906-1855-4bfa-8262-3f3dae458070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:05.071308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edc906-1855-4bfa-8262-3f3dae458070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:05.071641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5edc906-1855-4bfa-8262-3f3dae458070 admin admin] PATH_INFO: `/v3/users/953e22189f6347c486717da01015f6f0` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:05.074911 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5edc906-1855-4bfa-8262-3f3dae458070 admin admin] RBAC: Authorizing `identity:delete_user(user_id=953e22189f6347c486717da01015f6f0)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:05.080748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5edc906-1855-4bfa-8262-3f3dae458070 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:05.229861 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c5edc906-1855-4bfa-8262-3f3dae458070 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '953e22189f6347c486717da01015f6f0'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:05.231397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c5edc906-1855-4bfa-8262-3f3dae458070 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '953e22189f6347c486717da01015f6f0'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:05.237100 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c5edc906-1855-4bfa-8262-3f3dae458070 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '953e22189f6347c486717da01015f6f0'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:05.270200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c5edc906-1855-4bfa-8262-3f3dae458070 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '953e22189f6347c486717da01015f6f0'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:05.293302 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c5edc906-1855-4bfa-8262-3f3dae458070 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '953e22189f6347c486717da01015f6f0'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:05.318511 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 830/1673] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:04 2022] DELETE /identity/v3/users/953e22189f6347c486717da01015f6f0 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:05.323840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8eaf4-fd58-48de-9878-9ce7ebcac9f1 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:05.324819 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8eaf4-fd58-48de-9878-9ce7ebcac9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:05.325356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd8eaf4-fd58-48de-9878-9ce7ebcac9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:05.375823 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5edc906-1855-4bfa-8262-3f3dae458070 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:05.377396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-c5edc906-1855-4bfa-8262-3f3dae458070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:05.377604 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-dcd8eaf4-fd58-48de-9878-9ce7ebcac9f1 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:05.378814 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-dcd8eaf4-fd58-48de-9878-9ce7ebcac9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:05.488141 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:05.512333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5edc906-1855-4bfa-8262-3f3dae458070 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:05.540566 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b1a56-e691-4b81-aa37-c76f923e5a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:05.540996 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b1a56-e691-4b81-aa37-c76f923e5a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:05.542451 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b1a56-e691-4b81-aa37-c76f923e5a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:05.542898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b1a56-e691-4b81-aa37-c76f923e5a9a admin admin] PATH_INFO: `/v3/projects/1ccd522731d34716b2f806e4e51ba8e3` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:05.545785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b1a56-e691-4b81-aa37-c76f923e5a9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ccd522731d34716b2f806e4e51ba8e3)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:05.547356 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b1a56-e691-4b81-aa37-c76f923e5a9a admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:05.617147 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 844/1674] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:05.680445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c99b1a56-e691-4b81-aa37-c76f923e5a9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ccd522731d34716b2f806e4e51ba8e3', 'request_id': 'req-c99b1a56-e691-4b81-aa37-c76f923e5a9a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:05.702668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c99b1a56-e691-4b81-aa37-c76f923e5a9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ccd522731d34716b2f806e4e51ba8e3', 'request_id': 'req-c99b1a56-e691-4b81-aa37-c76f923e5a9a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:05.709539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c99b1a56-e691-4b81-aa37-c76f923e5a9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ccd522731d34716b2f806e4e51ba8e3', 'request_id': 'req-c99b1a56-e691-4b81-aa37-c76f923e5a9a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:05.735278 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c99b1a56-e691-4b81-aa37-c76f923e5a9a admin admin] The token cache is being invalidate because project 1ccd522731d34716b2f806e4e51ba8e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:05.735799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c99b1a56-e691-4b81-aa37-c76f923e5a9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:05.737032 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c99b1a56-e691-4b81-aa37-c76f923e5a9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:05.747094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 831/1675] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:05 2022] DELETE /identity/v3/projects/1ccd522731d34716b2f806e4e51ba8e3 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:05.753214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c3d60-120c-47b6-8068-28b3df955271 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:05.753848 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c3d60-120c-47b6-8068-28b3df955271 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:05.754344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c3d60-120c-47b6-8068-28b3df955271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:05.758148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b1a56-e691-4b81-aa37-c76f923e5a9a admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:05.760855 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-c99b1a56-e691-4b81-aa37-c76f923e5a9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:05.819860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-a62c3d60-120c-47b6-8068-28b3df955271 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:05.833514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a62c3d60-120c-47b6-8068-28b3df955271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:05.946944 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b1a56-e691-4b81-aa37-c76f923e5a9a admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:05.951208 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a139ba8-c238-4c8e-a32f-99ba97fa12d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:05.951876 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a139ba8-c238-4c8e-a32f-99ba97fa12d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:05.952155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a139ba8-c238-4c8e-a32f-99ba97fa12d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:05.952453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a139ba8-c238-4c8e-a32f-99ba97fa12d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:05.954391 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-0a139ba8-c238-4c8e-a32f-99ba97fa12d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:06.041551 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 845/1676] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:06.041551 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c3d60-120c-47b6-8068-28b3df955271 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:06.043581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a62c3d60-120c-47b6-8068-28b3df955271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:06.048863 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a139ba8-c238-4c8e-a32f-99ba97fa12d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:06.050336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a139ba8-c238-4c8e-a32f-99ba97fa12d5 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:06.067323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c3d60-120c-47b6-8068-28b3df955271 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:06.071421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 832/1677] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:17:06.072753 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a4732-b8b6-4d17-9577-2754a79c9dbb None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:06.073659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a4732-b8b6-4d17-9577-2754a79c9dbb None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:06.073659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a4732-b8b6-4d17-9577-2754a79c9dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:06.088513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c3555-da61-4a32-bf5d-3658202637c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:06.088513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c3555-da61-4a32-bf5d-3658202637c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:06.088999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c3555-da61-4a32-bf5d-3658202637c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:06.089397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c3555-da61-4a32-bf5d-3658202637c3 admin admin] PATH_INFO: `/v3/users/174d40665dff4994bf194842fde391ba` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:06.095249 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c3555-da61-4a32-bf5d-3658202637c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=174d40665dff4994bf194842fde391ba)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:06.097585 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c3555-da61-4a32-bf5d-3658202637c3 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:06.100641 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:06.120573 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-151a4732-b8b6-4d17-9577-2754a79c9dbb None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:06.121965 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-151a4732-b8b6-4d17-9577-2754a79c9dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:06.134329 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 833/1678] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:06.135715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a4732-b8b6-4d17-9577-2754a79c9dbb None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:06.149923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a4732-b8b6-4d17-9577-2754a79c9dbb None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:06.158239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5498a-7e3e-4fba-970e-cab1149e2a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:06.158891 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5498a-7e3e-4fba-970e-cab1149e2a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:06.159172 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5498a-7e3e-4fba-970e-cab1149e2a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:06.159448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad5498a-7e3e-4fba-970e-cab1149e2a4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:06.160169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5498a-7e3e-4fba-970e-cab1149e2a4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:17:06.160609 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5498a-7e3e-4fba-970e-cab1149e2a4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:06.162308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad5498a-7e3e-4fba-970e-cab1149e2a4d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:06.197931 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 834/1679] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:17:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:17:06.202644 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5498a-7e3e-4fba-970e-cab1149e2a4d admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:06.219456 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-485c3555-da61-4a32-bf5d-3658202637c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '174d40665dff4994bf194842fde391ba'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:06.229272 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad5498a-7e3e-4fba-970e-cab1149e2a4d admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:06.234987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce19f7f-31ad-4bdd-87ee-9cf481a91b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:06.235873 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce19f7f-31ad-4bdd-87ee-9cf481a91b90 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:06.239573 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce19f7f-31ad-4bdd-87ee-9cf481a91b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:06.240068 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce19f7f-31ad-4bdd-87ee-9cf481a91b90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:06.241781 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce19f7f-31ad-4bdd-87ee-9cf481a91b90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:06.246580 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce19f7f-31ad-4bdd-87ee-9cf481a91b90 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:06.253722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-485c3555-da61-4a32-bf5d-3658202637c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '174d40665dff4994bf194842fde391ba'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:06.257859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-485c3555-da61-4a32-bf5d-3658202637c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '174d40665dff4994bf194842fde391ba'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:06.286772 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:06.301414 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-485c3555-da61-4a32-bf5d-3658202637c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '174d40665dff4994bf194842fde391ba'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:06.306614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-485c3555-da61-4a32-bf5d-3658202637c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '174d40665dff4994bf194842fde391ba'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:06.317939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 835/1680] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:06 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:06.323703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce19f7f-31ad-4bdd-87ee-9cf481a91b90 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:06.329820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-0ce19f7f-31ad-4bdd-87ee-9cf481a91b90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:06.336590 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 846/1681] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:06 2022] DELETE /identity/v3/users/174d40665dff4994bf194842fde391ba => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:06.344748 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb271612-ec92-4369-af19-463307afa1c3 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:06.345290 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb271612-ec92-4369-af19-463307afa1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:06.345544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb271612-ec92-4369-af19-463307afa1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:06.409307 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-cb271612-ec92-4369-af19-463307afa1c3 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:06.412100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-cb271612-ec92-4369-af19-463307afa1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:06.549865 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce19f7f-31ad-4bdd-87ee-9cf481a91b90 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:06.607096 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 847/1682] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:06.654400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672522f5-4bc5-403e-a707-c9077e1b1296 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:06.655187 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672522f5-4bc5-403e-a707-c9077e1b1296 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:06.655815 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672522f5-4bc5-403e-a707-c9077e1b1296 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:06.662139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 848/1683] 10.222.0.45 () {54 vars in 961 bytes} [Thu Aug 4 02:17:06 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 04 02:17:06.670356 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a882c8e-9acd-48a3-9a53-8bec1c0acf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:06.671013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a882c8e-9acd-48a3-9a53-8bec1c0acf40 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:06.671300 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a882c8e-9acd-48a3-9a53-8bec1c0acf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:06.671575 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a882c8e-9acd-48a3-9a53-8bec1c0acf40 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:06.672256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a882c8e-9acd-48a3-9a53-8bec1c0acf40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:06.674276 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a882c8e-9acd-48a3-9a53-8bec1c0acf40 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:06.675752 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a425f-1706-4b5c-97fb-f702376a277a None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:06.676645 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a425f-1706-4b5c-97fb-f702376a277a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:06.677679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a425f-1706-4b5c-97fb-f702376a277a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:06.717730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 836/1684] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:06 2022] POST /identity/v3/users => generated 288 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:06.722838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a882c8e-9acd-48a3-9a53-8bec1c0acf40 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:06.737544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2a882c8e-9acd-48a3-9a53-8bec1c0acf40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:06.743734 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-ed9a425f-1706-4b5c-97fb-f702376a277a None None] MFA Rules not processed for user `796772cf491047c8b0a609dae9402307`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:06.747549 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-ed9a425f-1706-4b5c-97fb-f702376a277a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:06.886186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a882c8e-9acd-48a3-9a53-8bec1c0acf40 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:06.891146 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 849/1685] 10.222.0.45 () {58 vars in 1058 bytes} [Thu Aug 4 02:17:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:06.893148 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a425f-1706-4b5c-97fb-f702376a277a None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:06.899288 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075a999-874a-4a13-9fdb-1f9c8d56c77d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:06.900116 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075a999-874a-4a13-9fdb-1f9c8d56c77d service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:06.901271 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075a999-874a-4a13-9fdb-1f9c8d56c77d service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:06.901740 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075a999-874a-4a13-9fdb-1f9c8d56c77d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:06.919783 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a425f-1706-4b5c-97fb-f702376a277a None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:06.932517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469a56c-712e-4ee1-b199-17d92db00015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:06.934401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469a56c-712e-4ee1-b199-17d92db00015 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:06.934692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469a56c-712e-4ee1-b199-17d92db00015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:06.934965 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469a56c-712e-4ee1-b199-17d92db00015 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:06.936960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469a56c-712e-4ee1-b199-17d92db00015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:06.942008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469a56c-712e-4ee1-b199-17d92db00015 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:06.948300 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075a999-874a-4a13-9fdb-1f9c8d56c77d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:06.949946 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075a999-874a-4a13-9fdb-1f9c8d56c77d service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:06.976776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 850/1686] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:06.978723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469a56c-712e-4ee1-b199-17d92db00015 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:06.992087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 837/1687] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:06.994467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075a999-874a-4a13-9fdb-1f9c8d56c77d service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:07.009800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469a56c-712e-4ee1-b199-17d92db00015 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:07.025356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a58f8e-f8d9-4bbb-b83d-c09b87467fd2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796772cf491047c8b0a609dae9402307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:07.026002 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a58f8e-f8d9-4bbb-b83d-c09b87467fd2 service placement] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:07.026279 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a58f8e-f8d9-4bbb-b83d-c09b87467fd2 service placement] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:07.026553 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a58f8e-f8d9-4bbb-b83d-c09b87467fd2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:07.028664 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075a999-874a-4a13-9fdb-1f9c8d56c77d service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:07.032401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-37a58f8e-f8d9-4bbb-b83d-c09b87467fd2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:07.045765 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8833456-338e-48f7-b176-82f80695c4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:07.046775 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8833456-338e-48f7-b176-82f80695c4c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:07.050615 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:07.051261 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8833456-338e-48f7-b176-82f80695c4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:07.051775 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8833456-338e-48f7-b176-82f80695c4c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:07.052747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8833456-338e-48f7-b176-82f80695c4c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:07.055233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8833456-338e-48f7-b176-82f80695c4c6 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:07.062590 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a58f8e-f8d9-4bbb-b83d-c09b87467fd2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:07.064474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a58f8e-f8d9-4bbb-b83d-c09b87467fd2 service placement] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:07.076576 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 838/1688] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:07.081242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8833456-338e-48f7-b176-82f80695c4c6 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:07.101717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8833456-338e-48f7-b176-82f80695c4c6 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:07.102544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 851/1689] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:07.117083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86d228-f274-4654-b73b-57cd1bf76dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:07.118954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86d228-f274-4654-b73b-57cd1bf76dfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:07.119245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86d228-f274-4654-b73b-57cd1bf76dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:07.119657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86d228-f274-4654-b73b-57cd1bf76dfa admin admin] PATH_INFO: `/v3/projects/e5ba8efdcf7f43fe91a00cfae0fb12bd/users/97a6162bf9714d6eb101896535fc71f3/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:07.143644 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86d228-f274-4654-b73b-57cd1bf76dfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ba8efdcf7f43fe91a00cfae0fb12bd, user_id=97a6162bf9714d6eb101896535fc71f3, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:07.146549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86d228-f274-4654-b73b-57cd1bf76dfa admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:07.164591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:07.179898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 839/1690] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:07 2022] PUT /identity/v3/projects/e5ba8efdcf7f43fe91a00cfae0fb12bd/users/97a6162bf9714d6eb101896535fc71f3/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:07.192910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc58840-31ac-46e6-bdc3-5241a877e0c7 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:07.193198 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc58840-31ac-46e6-bdc3-5241a877e0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:07.193472 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc58840-31ac-46e6-bdc3-5241a877e0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:07.271927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-7cc58840-31ac-46e6-bdc3-5241a877e0c7 None None] MFA Rules not processed for user `97a6162bf9714d6eb101896535fc71f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:07.273406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7cc58840-31ac-46e6-bdc3-5241a877e0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:07.494182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 852/1691] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:07.499443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86d228-f274-4654-b73b-57cd1bf76dfa admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:07.595687 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae51b88-741c-4ec1-ba90-5998aa4231de None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:07.596037 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae51b88-741c-4ec1-ba90-5998aa4231de None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:07.596255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae51b88-741c-4ec1-ba90-5998aa4231de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:07.753991 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-6ae51b88-741c-4ec1-ba90-5998aa4231de None None] MFA Rules not processed for user `44655d95949b489088dbe48eeb8f38f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:07.761266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-6ae51b88-741c-4ec1-ba90-5998aa4231de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:07.781840 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86d228-f274-4654-b73b-57cd1bf76dfa admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:07.811387 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc7e7af-11c7-480b-bc96-ba8441280445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:07.812030 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7e7af-11c7-480b-bc96-ba8441280445 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:07.812307 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7e7af-11c7-480b-bc96-ba8441280445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:07.812594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc7e7af-11c7-480b-bc96-ba8441280445 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:07.813282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7e7af-11c7-480b-bc96-ba8441280445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:07.815226 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc7e7af-11c7-480b-bc96-ba8441280445 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:07.919011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 840/1692] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:07 2022] POST /identity/v3/projects => generated 375 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:07.934636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4dcb6-8402-4a9a-9b8f-b67f5c241f62 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:07.942277 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4dcb6-8402-4a9a-9b8f-b67f5c241f62 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:07.944278 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4dcb6-8402-4a9a-9b8f-b67f5c241f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:08.015960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 853/1693] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:08.029266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae51b88-741c-4ec1-ba90-5998aa4231de None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:08.051421 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae51b88-741c-4ec1-ba90-5998aa4231de None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:08.055072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-19d4dcb6-8402-4a9a-9b8f-b67f5c241f62 None None] MFA Rules not processed for user `033f13b2943945d3abc4ff852b68d698`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:08.057705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-19d4dcb6-8402-4a9a-9b8f-b67f5c241f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:08.231544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e3153-dd7f-4f51-bddb-ca4a9e518c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:08.232199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e3153-dd7f-4f51-bddb-ca4a9e518c34 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:08.232481 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e3153-dd7f-4f51-bddb-ca4a9e518c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:08.232784 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e3153-dd7f-4f51-bddb-ca4a9e518c34 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:08.233628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e3153-dd7f-4f51-bddb-ca4a9e518c34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:08.235602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e3153-dd7f-4f51-bddb-ca4a9e518c34 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:08.298977 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 854/1694] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:08 2022] POST /identity/v3/users => generated 294 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:08.300333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6365a6-439f-43bc-a3f1-c6933f3b46fe None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:08.300706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6365a6-439f-43bc-a3f1-c6933f3b46fe None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:08.300926 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6365a6-439f-43bc-a3f1-c6933f3b46fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:08.336985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:08.363444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 841/1695] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:08.366540 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4dcb6-8402-4a9a-9b8f-b67f5c241f62 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:08.374751 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-9a6365a6-439f-43bc-a3f1-c6933f3b46fe None None] MFA Rules not processed for user `44655d95949b489088dbe48eeb8f38f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:08.376320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9a6365a6-439f-43bc-a3f1-c6933f3b46fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:08.409076 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4dcb6-8402-4a9a-9b8f-b67f5c241f62 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:08.443256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05263bc0-17e4-4bfa-b6f6-58953e98ca5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:08.446019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05263bc0-17e4-4bfa-b6f6-58953e98ca5c admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:08.446309 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05263bc0-17e4-4bfa-b6f6-58953e98ca5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:08.449178 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05263bc0-17e4-4bfa-b6f6-58953e98ca5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:08.452148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05263bc0-17e4-4bfa-b6f6-58953e98ca5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:08.459315 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05263bc0-17e4-4bfa-b6f6-58953e98ca5c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:08.524744 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 842/1696] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:08.535488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05263bc0-17e4-4bfa-b6f6-58953e98ca5c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:08.570779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 855/1697] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:08.573142 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05263bc0-17e4-4bfa-b6f6-58953e98ca5c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:08.582102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6365a6-439f-43bc-a3f1-c6933f3b46fe None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:08.583540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9a6365a6-439f-43bc-a3f1-c6933f3b46fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:08.598435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f5b78-4925-448b-83e9-88b5a6599a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:08.598942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f5b78-4925-448b-83e9-88b5a6599a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:08.599190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f5b78-4925-448b-83e9-88b5a6599a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:08.599470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f5b78-4925-448b-83e9-88b5a6599a0b admin admin] PATH_INFO: `/v3/projects/6d02265f564e46f4b0d1cd2a42c9d167/users/7c745d50ea8a43df90e73380695d3c32/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:08.668587 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f5b78-4925-448b-83e9-88b5a6599a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d02265f564e46f4b0d1cd2a42c9d167, user_id=7c745d50ea8a43df90e73380695d3c32, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:08.692298 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f5b78-4925-448b-83e9-88b5a6599a0b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:08.737023 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 843/1698] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:08 2022] PUT /identity/v3/projects/6d02265f564e46f4b0d1cd2a42c9d167/users/7c745d50ea8a43df90e73380695d3c32/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:08.742641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f5b78-4925-448b-83e9-88b5a6599a0b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:08.846005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6365a6-439f-43bc-a3f1-c6933f3b46fe None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:08.925081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:08.985549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f5b78-4925-448b-83e9-88b5a6599a0b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:09.009722 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909da505-171b-437d-990f-913320869885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:09.010384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909da505-171b-437d-990f-913320869885 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:09.010668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909da505-171b-437d-990f-913320869885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:09.010943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909da505-171b-437d-990f-913320869885 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:09.011664 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909da505-171b-437d-990f-913320869885 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:09.013360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909da505-171b-437d-990f-913320869885 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:09.062187 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 844/1699] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:09.066677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909da505-171b-437d-990f-913320869885 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:09.082338 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6daee-8b3c-4b80-b288-7d8934c98932 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:09.083069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6daee-8b3c-4b80-b288-7d8934c98932 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:09.083362 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6daee-8b3c-4b80-b288-7d8934c98932 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:09.083641 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6daee-8b3c-4b80-b288-7d8934c98932 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:09.108553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909da505-171b-437d-990f-913320869885 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:09.128360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099ee7c-f763-4d1e-8a19-ff1f93a0f679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:09.131394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099ee7c-f763-4d1e-8a19-ff1f93a0f679 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:09.131740 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099ee7c-f763-4d1e-8a19-ff1f93a0f679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:09.132027 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099ee7c-f763-4d1e-8a19-ff1f93a0f679 admin admin] PATH_INFO: `/v3/projects/6d02265f564e46f4b0d1cd2a42c9d167/users/7c745d50ea8a43df90e73380695d3c32/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:09.141954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099ee7c-f763-4d1e-8a19-ff1f93a0f679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d02265f564e46f4b0d1cd2a42c9d167, user_id=7c745d50ea8a43df90e73380695d3c32, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:09.149155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099ee7c-f763-4d1e-8a19-ff1f93a0f679 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:09.213353 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 845/1700] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:09 2022] PUT /identity/v3/projects/6d02265f564e46f4b0d1cd2a42c9d167/users/7c745d50ea8a43df90e73380695d3c32/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:09.216426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb005eb-1ee8-4050-a325-160693e666de None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:09.217336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb005eb-1ee8-4050-a325-160693e666de None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:09.217628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb005eb-1ee8-4050-a325-160693e666de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:09.219771 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:09.264548 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6daee-8b3c-4b80-b288-7d8934c98932 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:09.266199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6daee-8b3c-4b80-b288-7d8934c98932 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:09.280869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 856/1701] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 699 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:09.327505 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-bfb005eb-1ee8-4050-a325-160693e666de None None] MFA Rules not processed for user `7c745d50ea8a43df90e73380695d3c32`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:09.332410 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-bfb005eb-1ee8-4050-a325-160693e666de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:09.412571 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6daee-8b3c-4b80-b288-7d8934c98932 service nova] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:09.415482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-15f6daee-8b3c-4b80-b288-7d8934c98932 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:09.453406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6daee-8b3c-4b80-b288-7d8934c98932 service nova] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:09.598966 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 846/1702] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:09 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:09.600953 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb005eb-1ee8-4050-a325-160693e666de None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:09.603565 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-bfb005eb-1ee8-4050-a325-160693e666de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:09.751503 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41c2fc-642d-428d-930d-7837d2da54d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:09.755954 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41c2fc-642d-428d-930d-7837d2da54d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:09.757886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41c2fc-642d-428d-930d-7837d2da54d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:09.758287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41c2fc-642d-428d-930d-7837d2da54d3 admin admin] PATH_INFO: `/v3/projects/280553ef422943e3b37e63b7adc9ded2` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:09.760471 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41c2fc-642d-428d-930d-7837d2da54d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=280553ef422943e3b37e63b7adc9ded2)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:09.763102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41c2fc-642d-428d-930d-7837d2da54d3 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:09.814458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb005eb-1ee8-4050-a325-160693e666de None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:09.825700 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a64b8-62e9-48c2-b0dc-0a3bba06e123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:09.829460 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a64b8-62e9-48c2-b0dc-0a3bba06e123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:09.830144 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a64b8-62e9-48c2-b0dc-0a3bba06e123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:09.831338 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562a64b8-62e9-48c2-b0dc-0a3bba06e123 admin admin] PATH_INFO: `/v3/projects/babea4c173ce42f49f43d8b09468b1e0` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:09.835088 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a64b8-62e9-48c2-b0dc-0a3bba06e123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=babea4c173ce42f49f43d8b09468b1e0)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:09.841581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562a64b8-62e9-48c2-b0dc-0a3bba06e123 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:09.958773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-df41c2fc-642d-428d-930d-7837d2da54d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '280553ef422943e3b37e63b7adc9ded2', 'request_id': 'req-df41c2fc-642d-428d-930d-7837d2da54d3'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:09.970665 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-562a64b8-62e9-48c2-b0dc-0a3bba06e123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'babea4c173ce42f49f43d8b09468b1e0', 'request_id': 'req-562a64b8-62e9-48c2-b0dc-0a3bba06e123'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:09.976789 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-df41c2fc-642d-428d-930d-7837d2da54d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '280553ef422943e3b37e63b7adc9ded2', 'request_id': 'req-df41c2fc-642d-428d-930d-7837d2da54d3'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:09.987829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-562a64b8-62e9-48c2-b0dc-0a3bba06e123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'babea4c173ce42f49f43d8b09468b1e0', 'request_id': 'req-562a64b8-62e9-48c2-b0dc-0a3bba06e123'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:09.994421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:09.996606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-562a64b8-62e9-48c2-b0dc-0a3bba06e123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'babea4c173ce42f49f43d8b09468b1e0', 'request_id': 'req-562a64b8-62e9-48c2-b0dc-0a3bba06e123'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:09.997424 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-df41c2fc-642d-428d-930d-7837d2da54d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '280553ef422943e3b37e63b7adc9ded2', 'request_id': 'req-df41c2fc-642d-428d-930d-7837d2da54d3'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:10.019781 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-562a64b8-62e9-48c2-b0dc-0a3bba06e123 admin admin] The token cache is being invalidate because project babea4c173ce42f49f43d8b09468b1e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:10.020254 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-562a64b8-62e9-48c2-b0dc-0a3bba06e123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:10.021195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-562a64b8-62e9-48c2-b0dc-0a3bba06e123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:10.026351 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 847/1703] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:09 2022] DELETE /identity/v3/projects/babea4c173ce42f49f43d8b09468b1e0 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:10.031643 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a64b8-62e9-48c2-b0dc-0a3bba06e123 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:10.033881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-562a64b8-62e9-48c2-b0dc-0a3bba06e123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:10.038602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-df41c2fc-642d-428d-930d-7837d2da54d3 admin admin] The token cache is being invalidate because project 280553ef422943e3b37e63b7adc9ded2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:10.038902 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-df41c2fc-642d-428d-930d-7837d2da54d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:10.040264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-df41c2fc-642d-428d-930d-7837d2da54d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:10.051345 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 857/1704] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:09 2022] DELETE /identity/v3/projects/280553ef422943e3b37e63b7adc9ded2 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:10.051345 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41c2fc-642d-428d-930d-7837d2da54d3 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:10.052465 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-df41c2fc-642d-428d-930d-7837d2da54d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:10.173580 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562a64b8-62e9-48c2-b0dc-0a3bba06e123 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:10.180241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41c2fc-642d-428d-930d-7837d2da54d3 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:10.186927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360cd3d0-1059-4723-bb6a-aa92f39ca2ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:10.190087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cd3d0-1059-4723-bb6a-aa92f39ca2ea service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:10.191941 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cd3d0-1059-4723-bb6a-aa92f39ca2ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:10.192382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360cd3d0-1059-4723-bb6a-aa92f39ca2ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:10.198988 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-360cd3d0-1059-4723-bb6a-aa92f39ca2ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:10.205298 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:10.308422 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1336c251-56e5-4849-9b72-07e119c54504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:10.309234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336c251-56e5-4849-9b72-07e119c54504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:10.309367 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336c251-56e5-4849-9b72-07e119c54504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:10.309649 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1336c251-56e5-4849-9b72-07e119c54504 admin admin] PATH_INFO: `/v3/users/9350726e88784a96905a89b7762d34a5` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:10.311754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336c251-56e5-4849-9b72-07e119c54504 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9350726e88784a96905a89b7762d34a5)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:10.317237 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1336c251-56e5-4849-9b72-07e119c54504 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:10.413536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360cd3d0-1059-4723-bb6a-aa92f39ca2ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:10.421346 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360cd3d0-1059-4723-bb6a-aa92f39ca2ea service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:10.440855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1336c251-56e5-4849-9b72-07e119c54504 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9350726e88784a96905a89b7762d34a5'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:10.453194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1336c251-56e5-4849-9b72-07e119c54504 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9350726e88784a96905a89b7762d34a5'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:10.454587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1336c251-56e5-4849-9b72-07e119c54504 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9350726e88784a96905a89b7762d34a5'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:10.460706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 848/1705] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:10.465312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d70a8-dad3-4df3-999c-9886691f40e4 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:10.465777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d70a8-dad3-4df3-999c-9886691f40e4 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:10.466216 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409d70a8-dad3-4df3-999c-9886691f40e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:10.473986 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1336c251-56e5-4849-9b72-07e119c54504 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9350726e88784a96905a89b7762d34a5'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:10.475057 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1336c251-56e5-4849-9b72-07e119c54504 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9350726e88784a96905a89b7762d34a5'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:10.495586 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 858/1706] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:10 2022] DELETE /identity/v3/users/9350726e88784a96905a89b7762d34a5 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:10.524056 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-409d70a8-dad3-4df3-999c-9886691f40e4 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:10.529029 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-409d70a8-dad3-4df3-999c-9886691f40e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:10.677237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 849/1707] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:10.685566 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794594a7-7cb3-4071-a495-45356aeb2341 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:10.686231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794594a7-7cb3-4071-a495-45356aeb2341 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:10.687004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794594a7-7cb3-4071-a495-45356aeb2341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:10.756379 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-794594a7-7cb3-4071-a495-45356aeb2341 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:10.759794 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-794594a7-7cb3-4071-a495-45356aeb2341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:10.784697 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 859/1708] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:10.790181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d70a8-dad3-4df3-999c-9886691f40e4 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:10.811205 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409d70a8-dad3-4df3-999c-9886691f40e4 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:10.847409 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf57ef0-d589-4f7e-81e2-6dadbefa3721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:10.858035 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf57ef0-d589-4f7e-81e2-6dadbefa3721 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:10.858366 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf57ef0-d589-4f7e-81e2-6dadbefa3721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:10.858650 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf57ef0-d589-4f7e-81e2-6dadbefa3721 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:10.863506 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf57ef0-d589-4f7e-81e2-6dadbefa3721 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:17:10.864124 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf57ef0-d589-4f7e-81e2-6dadbefa3721 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:10.869036 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf57ef0-d589-4f7e-81e2-6dadbefa3721 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:10.928843 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 850/1709] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:17:10.929902 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794594a7-7cb3-4071-a495-45356aeb2341 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:10.952663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2fc7-75c5-47a7-93e9-70368d10823d None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:10.953001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2fc7-75c5-47a7-93e9-70368d10823d None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:10.953287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b2fc7-75c5-47a7-93e9-70368d10823d None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:10.956559 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794594a7-7cb3-4071-a495-45356aeb2341 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:10.958233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 851/1710] 10.222.0.45 () {54 vars in 912 bytes} [Thu Aug 4 02:17:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:17:10.972411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d76ee6-a984-4372-bf71-e063e8a6392f None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:10.972749 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d76ee6-a984-4372-bf71-e063e8a6392f None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:10.974299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d76ee6-a984-4372-bf71-e063e8a6392f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:11.007930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2236e-ede2-489e-b52e-f7e4f3ad6c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:11.011789 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2236e-ede2-489e-b52e-f7e4f3ad6c3a admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:11.012089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2236e-ede2-489e-b52e-f7e4f3ad6c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:11.012585 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2236e-ede2-489e-b52e-f7e4f3ad6c3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:11.013441 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2236e-ede2-489e-b52e-f7e4f3ad6c3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:11.030583 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2236e-ede2-489e-b52e-f7e4f3ad6c3a admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:11.036403 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-14d76ee6-a984-4372-bf71-e063e8a6392f None None] MFA Rules not processed for user `6c2da427acb24c8cb6d667c6347858c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:11.039507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-14d76ee6-a984-4372-bf71-e063e8a6392f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:11.133366 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 860/1711] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:10 2022] POST /identity/v3/projects => generated 373 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:11.151031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2236e-ede2-489e-b52e-f7e4f3ad6c3a admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:11.153925 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:11.221636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 852/1712] 10.222.0.45 () {58 vars in 1009 bytes} [Thu Aug 4 02:17:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:11.346233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d76ee6-a984-4372-bf71-e063e8a6392f None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:11.347952 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-14d76ee6-a984-4372-bf71-e063e8a6392f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:11.357130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2236e-ede2-489e-b52e-f7e4f3ad6c3a admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:11.370348 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d76ee6-a984-4372-bf71-e063e8a6392f None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:11.378726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25117e54-6235-4128-b0a7-29f7e9827b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:11.378726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25117e54-6235-4128-b0a7-29f7e9827b83 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:11.378726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25117e54-6235-4128-b0a7-29f7e9827b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:11.378726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25117e54-6235-4128-b0a7-29f7e9827b83 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:11.379720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25117e54-6235-4128-b0a7-29f7e9827b83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:11.382543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25117e54-6235-4128-b0a7-29f7e9827b83 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:11.420647 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:11.433731 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 861/1713] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:11 2022] POST /identity/v3/users => generated 287 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:11.438022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25117e54-6235-4128-b0a7-29f7e9827b83 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:11.460849 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25117e54-6235-4128-b0a7-29f7e9827b83 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:11.467782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4de3a-a6f6-4649-9041-a59cb556ee71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:11.468421 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4de3a-a6f6-4649-9041-a59cb556ee71 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:11.468790 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4de3a-a6f6-4649-9041-a59cb556ee71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:11.469074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4de3a-a6f6-4649-9041-a59cb556ee71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:11.470000 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4de3a-a6f6-4649-9041-a59cb556ee71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:11.471761 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4de3a-a6f6-4649-9041-a59cb556ee71 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:11.492783 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 862/1714] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:11.496096 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd0c9c-227d-47c6-ba48-c45c72260685 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:11.496391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd0c9c-227d-47c6-ba48-c45c72260685 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:11.496714 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd0c9c-227d-47c6-ba48-c45c72260685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:11.521065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c9237-fa8c-4229-9072-e15912a20d18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:11.523189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c9237-fa8c-4229-9072-e15912a20d18 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:11.524192 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c9237-fa8c-4229-9072-e15912a20d18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:11.524527 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123c9237-fa8c-4229-9072-e15912a20d18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:11.548320 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c9237-fa8c-4229-9072-e15912a20d18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:11.554519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123c9237-fa8c-4229-9072-e15912a20d18 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:11.584098 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 853/1715] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:11.587654 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c9237-fa8c-4229-9072-e15912a20d18 service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:11.593130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-afdd0c9c-227d-47c6-ba48-c45c72260685 None None] MFA Rules not processed for user `e54e713b52e84393b018807f8153922e`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:11.594659 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-afdd0c9c-227d-47c6-ba48-c45c72260685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:11.614132 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123c9237-fa8c-4229-9072-e15912a20d18 service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:11.626955 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8106a-76a4-448a-9c99-bca594386f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:11.627672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8106a-76a4-448a-9c99-bca594386f4a admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:11.627953 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8106a-76a4-448a-9c99-bca594386f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:11.628372 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b8106a-76a4-448a-9c99-bca594386f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:11.629107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8106a-76a4-448a-9c99-bca594386f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:11.630736 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b8106a-76a4-448a-9c99-bca594386f4a admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:11.669412 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 854/1716] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:11 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:11.670231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8106a-76a4-448a-9c99-bca594386f4a admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:11.694228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b8106a-76a4-448a-9c99-bca594386f4a admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:11.703272 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e047b-8c2e-42ae-b533-9a0ccd97b17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:11.704288 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e047b-8c2e-42ae-b533-9a0ccd97b17a admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:11.704824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e047b-8c2e-42ae-b533-9a0ccd97b17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:11.705425 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e047b-8c2e-42ae-b533-9a0ccd97b17a admin admin] PATH_INFO: `/v3/projects/252324095c9b440dad83360a6b37b2cc/users/4e689db6d2d6493c86fe408a5160033d/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:11.730343 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e047b-8c2e-42ae-b533-9a0ccd97b17a admin admin] RBAC: Authorizing `identity:create_grant(project_id=252324095c9b440dad83360a6b37b2cc, user_id=4e689db6d2d6493c86fe408a5160033d, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:11.731551 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 863/1717] 10.222.0.45 () {58 vars in 1009 bytes} [Thu Aug 4 02:17:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:11.734685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e047b-8c2e-42ae-b533-9a0ccd97b17a admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:11.766923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 855/1718] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:11 2022] PUT /identity/v3/projects/252324095c9b440dad83360a6b37b2cc/users/4e689db6d2d6493c86fe408a5160033d/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:11.779481 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576336a-4ea5-4a00-bfeb-78bada2d3ff5 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:11.779982 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576336a-4ea5-4a00-bfeb-78bada2d3ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:11.780426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e576336a-4ea5-4a00-bfeb-78bada2d3ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:11.818515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203b8f2-1f39-4024-bb1d-784fb6c701fd None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:11.819410 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203b8f2-1f39-4024-bb1d-784fb6c701fd None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:11.819840 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1203b8f2-1f39-4024-bb1d-784fb6c701fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:11.823533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-e576336a-4ea5-4a00-bfeb-78bada2d3ff5 None None] MFA Rules not processed for user `4e689db6d2d6493c86fe408a5160033d`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:11.825303 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e576336a-4ea5-4a00-bfeb-78bada2d3ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:11.875857 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-1203b8f2-1f39-4024-bb1d-784fb6c701fd None None] MFA Rules not processed for user `df5d3de1afa945c3b0873050eca06c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:11.877357 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-1203b8f2-1f39-4024-bb1d-784fb6c701fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:11.923896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 864/1719] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:11.925020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576336a-4ea5-4a00-bfeb-78bada2d3ff5 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:11.927507 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e576336a-4ea5-4a00-bfeb-78bada2d3ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:12.012780 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:12.098617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 856/1720] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:11 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:12.098617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203b8f2-1f39-4024-bb1d-784fb6c701fd None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:12.121023 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e576336a-4ea5-4a00-bfeb-78bada2d3ff5 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:12.127730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d12d9-104e-4d61-b812-6a9dbac9faa1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '155a7276efab426cb34175683c475f33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:12.128756 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d12d9-104e-4d61-b812-6a9dbac9faa1 service swift] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:12.129187 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d12d9-104e-4d61-b812-6a9dbac9faa1 service swift] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:12.129872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7d12d9-104e-4d61-b812-6a9dbac9faa1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:12.195744 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:12.236540 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1203b8f2-1f39-4024-bb1d-784fb6c701fd None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:12.243496 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3dd98-2b27-43c9-975e-e47d144fb8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:12.244714 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3dd98-2b27-43c9-975e-e47d144fb8c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:12.245602 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3dd98-2b27-43c9-975e-e47d144fb8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:12.246145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab3dd98-2b27-43c9-975e-e47d144fb8c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:12.247849 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3dd98-2b27-43c9-975e-e47d144fb8c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:12.251188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab3dd98-2b27-43c9-975e-e47d144fb8c1 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:12.268690 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d12d9-104e-4d61-b812-6a9dbac9faa1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:12.271578 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7d12d9-104e-4d61-b812-6a9dbac9faa1 service swift] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:12.297552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 865/1721] 10.222.0.45 () {60 vars in 1384 bytes} [Thu Aug 4 02:17:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 02:17:12.330268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 857/1722] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:12 2022] POST /identity/v3/projects => generated 373 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:12.334894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d12d9-104e-4d61-b812-6a9dbac9faa1 service swift] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:12.346632 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7d12d9-104e-4d61-b812-6a9dbac9faa1 service swift] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:12.513449 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c207e6b-8aca-48f7-9e36-ceb8f9798da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:12.514094 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c207e6b-8aca-48f7-9e36-ceb8f9798da9 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:12.514447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c207e6b-8aca-48f7-9e36-ceb8f9798da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:12.514816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c207e6b-8aca-48f7-9e36-ceb8f9798da9 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:12.515565 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c207e6b-8aca-48f7-9e36-ceb8f9798da9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:12.518070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c207e6b-8aca-48f7-9e36-ceb8f9798da9 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:12.609622 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3dd98-2b27-43c9-975e-e47d144fb8c1 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:12.610479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 866/1723] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:12 2022] POST /identity/v3/users => generated 293 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:12.626043 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab3dd98-2b27-43c9-975e-e47d144fb8c1 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:12.673370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c703bf8-8424-4ab4-a1eb-f557668ba8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:12.677365 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c703bf8-8424-4ab4-a1eb-f557668ba8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:12.677770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c703bf8-8424-4ab4-a1eb-f557668ba8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:12.678316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c703bf8-8424-4ab4-a1eb-f557668ba8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:12.678894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c703bf8-8424-4ab4-a1eb-f557668ba8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:12.688304 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c703bf8-8424-4ab4-a1eb-f557668ba8e7 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:12.772594 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c207e6b-8aca-48f7-9e36-ceb8f9798da9 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:12.786503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 858/1724] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:12.793648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c207e6b-8aca-48f7-9e36-ceb8f9798da9 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:12.818791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0a3c6-d653-4b90-bdc3-165df08f6f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:12.819653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0a3c6-d653-4b90-bdc3-165df08f6f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:12.819787 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0a3c6-d653-4b90-bdc3-165df08f6f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:12.820004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0a3c6-d653-4b90-bdc3-165df08f6f70 admin admin] PATH_INFO: `/v3/projects/2842783593654022b700982a818b3df2/users/fbb847bb36ea4e2fb5927669ff3a6cc7/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:12.902906 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0a3c6-d653-4b90-bdc3-165df08f6f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2842783593654022b700982a818b3df2, user_id=fbb847bb36ea4e2fb5927669ff3a6cc7, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:12.913919 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f0a3c6-d653-4b90-bdc3-165df08f6f70 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:12.961049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 867/1725] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:12 2022] PUT /identity/v3/projects/2842783593654022b700982a818b3df2/users/fbb847bb36ea4e2fb5927669ff3a6cc7/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:12.965228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c703bf8-8424-4ab4-a1eb-f557668ba8e7 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:12.982669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c703bf8-8424-4ab4-a1eb-f557668ba8e7 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:13.103117 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec8b7e-e79f-4f6c-9f37-cc1978a392ac None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:13.103485 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec8b7e-e79f-4f6c-9f37-cc1978a392ac None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:13.103771 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec8b7e-e79f-4f6c-9f37-cc1978a392ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:13.226146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd65bd0-402b-4a4f-a214-37520eb3813f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:13.232014 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd65bd0-402b-4a4f-a214-37520eb3813f admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:13.233500 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd65bd0-402b-4a4f-a214-37520eb3813f admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:13.234813 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd65bd0-402b-4a4f-a214-37520eb3813f admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:13.235733 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd65bd0-402b-4a4f-a214-37520eb3813f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:13.242172 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-7fec8b7e-e79f-4f6c-9f37-cc1978a392ac None None] MFA Rules not processed for user `76703dc82dc644ef8d32f7e259ddea6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:13.243419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7fec8b7e-e79f-4f6c-9f37-cc1978a392ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:13.245916 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd65bd0-402b-4a4f-a214-37520eb3813f admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:13.272914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 859/1726] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:13.274002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd65bd0-402b-4a4f-a214-37520eb3813f admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:13.281259 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-1fd65bd0-402b-4a4f-a214-37520eb3813f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:13.308968 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd65bd0-402b-4a4f-a214-37520eb3813f admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:13.321452 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f760aa9-d4a7-4d02-badd-d000509592f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:13.322674 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f760aa9-d4a7-4d02-badd-d000509592f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:13.322674 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f760aa9-d4a7-4d02-badd-d000509592f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:13.323083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f760aa9-d4a7-4d02-badd-d000509592f9 admin admin] PATH_INFO: `/v3/projects/d506cb539df146128fe763e275f29d03` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:13.323188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:13.324561 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f760aa9-d4a7-4d02-badd-d000509592f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d506cb539df146128fe763e275f29d03)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:13.331917 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f760aa9-d4a7-4d02-badd-d000509592f9 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:13.447953 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0f760aa9-d4a7-4d02-badd-d000509592f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd506cb539df146128fe763e275f29d03', 'request_id': 'req-0f760aa9-d4a7-4d02-badd-d000509592f9'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:13.465723 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0f760aa9-d4a7-4d02-badd-d000509592f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd506cb539df146128fe763e275f29d03', 'request_id': 'req-0f760aa9-d4a7-4d02-badd-d000509592f9'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:13.496082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 868/1727] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:13.498396 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec8b7e-e79f-4f6c-9f37-cc1978a392ac None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:13.504061 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0f760aa9-d4a7-4d02-badd-d000509592f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd506cb539df146128fe763e275f29d03', 'request_id': 'req-0f760aa9-d4a7-4d02-badd-d000509592f9'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:13.555805 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0f760aa9-d4a7-4d02-badd-d000509592f9 admin admin] The token cache is being invalidate because project d506cb539df146128fe763e275f29d03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:13.557414 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec8b7e-e79f-4f6c-9f37-cc1978a392ac None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:13.559021 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0f760aa9-d4a7-4d02-badd-d000509592f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:13.561976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0f760aa9-d4a7-4d02-badd-d000509592f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:13.574862 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 860/1728] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:13 2022] DELETE /identity/v3/projects/d506cb539df146128fe763e275f29d03 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:13.576535 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77352010-7410-44fc-96c9-00ee26786473 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:13.581397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77352010-7410-44fc-96c9-00ee26786473 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:13.583135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77352010-7410-44fc-96c9-00ee26786473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:13.704558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-77352010-7410-44fc-96c9-00ee26786473 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:13.706101 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-77352010-7410-44fc-96c9-00ee26786473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:13.846548 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b8359-db6a-4df0-aa37-0d658f2cceba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:13.848497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b8359-db6a-4df0-aa37-0d658f2cceba admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:13.848966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b8359-db6a-4df0-aa37-0d658f2cceba admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:13.853044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258b8359-db6a-4df0-aa37-0d658f2cceba admin admin] PATH_INFO: `/v3/projects/2842783593654022b700982a818b3df2/users/fbb847bb36ea4e2fb5927669ff3a6cc7/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:13.858823 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258b8359-db6a-4df0-aa37-0d658f2cceba admin admin] RBAC: Authorizing `identity:create_grant(project_id=2842783593654022b700982a818b3df2, user_id=fbb847bb36ea4e2fb5927669ff3a6cc7, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:13.872699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258b8359-db6a-4df0-aa37-0d658f2cceba admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:13.947821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 869/1729] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:13 2022] PUT /identity/v3/projects/2842783593654022b700982a818b3df2/users/fbb847bb36ea4e2fb5927669ff3a6cc7/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:13.949535 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b8359-db6a-4df0-aa37-0d658f2cceba admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:13.951508 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-258b8359-db6a-4df0-aa37-0d658f2cceba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:14.015262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:14.118196 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 861/1730] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 542 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:14.120117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1b939-97f1-4782-9023-128b4a9ac1e9 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:14.120411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1b939-97f1-4782-9023-128b4a9ac1e9 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:14.120718 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e1b939-97f1-4782-9023-128b4a9ac1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:14.217405 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-a4e1b939-97f1-4782-9023-128b4a9ac1e9 None None] MFA Rules not processed for user `fbb847bb36ea4e2fb5927669ff3a6cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:14.221855 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a4e1b939-97f1-4782-9023-128b4a9ac1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:14.383922 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258b8359-db6a-4df0-aa37-0d658f2cceba admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:14.402099 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d3771-271a-40fe-b628-3fea07ec8c03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:14.402781 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d3771-271a-40fe-b628-3fea07ec8c03 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:14.403152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d3771-271a-40fe-b628-3fea07ec8c03 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:14.403429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d3771-271a-40fe-b628-3fea07ec8c03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:14.433128 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f28d3771-271a-40fe-b628-3fea07ec8c03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:14.597247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 862/1731] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:14.599006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e356f-a24c-4785-8a04-3811eae0d757 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:14.602748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e356f-a24c-4785-8a04-3811eae0d757 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:14.603082 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133e356f-a24c-4785-8a04-3811eae0d757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:14.672297 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-133e356f-a24c-4785-8a04-3811eae0d757 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:14.680370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-133e356f-a24c-4785-8a04-3811eae0d757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:14.711206 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d3771-271a-40fe-b628-3fea07ec8c03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:14.734210 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d3771-271a-40fe-b628-3fea07ec8c03 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:14.769736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:14.793453 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 870/1732] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:13 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 844 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:14.795625 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d3771-271a-40fe-b628-3fea07ec8c03 service nova] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:14.807607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f28d3771-271a-40fe-b628-3fea07ec8c03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:15.037866 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 863/1733] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:15.041706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e356f-a24c-4785-8a04-3811eae0d757 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:15.044018 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-133e356f-a24c-4785-8a04-3811eae0d757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:15.090875 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133e356f-a24c-4785-8a04-3811eae0d757 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:15.092531 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d3771-271a-40fe-b628-3fea07ec8c03 service nova] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:15.107114 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29c7ed-dda2-4f4f-a711-e1d4884963d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:15.110237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29c7ed-dda2-4f4f-a711-e1d4884963d0 service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:15.111992 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29c7ed-dda2-4f4f-a711-e1d4884963d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:15.112451 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e29c7ed-dda2-4f4f-a711-e1d4884963d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:15.119248 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-4e29c7ed-dda2-4f4f-a711-e1d4884963d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:15.121769 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caea78d-8b84-414e-b8a9-7ab840c243d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:15.124792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caea78d-8b84-414e-b8a9-7ab840c243d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:15.125110 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caea78d-8b84-414e-b8a9-7ab840c243d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:15.128274 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caea78d-8b84-414e-b8a9-7ab840c243d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:15.170069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caea78d-8b84-414e-b8a9-7ab840c243d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:15.175306 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caea78d-8b84-414e-b8a9-7ab840c243d8 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:15.203294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 871/1734] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:15.203294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caea78d-8b84-414e-b8a9-7ab840c243d8 service cinder] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:15.230122 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caea78d-8b84-414e-b8a9-7ab840c243d8 service cinder] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:15.244007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1a0b6-043c-47b6-9905-5826c04c178f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:15.245238 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1a0b6-043c-47b6-9905-5826c04c178f admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:15.245238 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1a0b6-043c-47b6-9905-5826c04c178f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:15.245238 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1a0b6-043c-47b6-9905-5826c04c178f admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:15.245912 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1a0b6-043c-47b6-9905-5826c04c178f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:17:15.246221 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1a0b6-043c-47b6-9905-5826c04c178f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:15.247930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1a0b6-043c-47b6-9905-5826c04c178f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:15.281587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 872/1735] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:17:15.283040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1a0b6-043c-47b6-9905-5826c04c178f admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:15.313250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1a0b6-043c-47b6-9905-5826c04c178f admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:15.346692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67a442-8d14-4313-b954-b7e743cfbd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:15.347644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67a442-8d14-4313-b954-b7e743cfbd3f admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:15.347937 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67a442-8d14-4313-b954-b7e743cfbd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:15.356860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67a442-8d14-4313-b954-b7e743cfbd3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:15.356860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67a442-8d14-4313-b954-b7e743cfbd3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:15.356860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67a442-8d14-4313-b954-b7e743cfbd3f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:15.456752 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 873/1736] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:15 2022] POST /identity/v3/projects => generated 381 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:15.465224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67a442-8d14-4313-b954-b7e743cfbd3f admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:15.571024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29c7ed-dda2-4f4f-a711-e1d4884963d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:15.573711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e29c7ed-dda2-4f4f-a711-e1d4884963d0 service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:15.599895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:15.668112 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 864/1737] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:15.693061 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67a442-8d14-4313-b954-b7e743cfbd3f admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:15.728905 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b628582-ea30-4b13-90ef-b70909f052fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:15.733118 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b628582-ea30-4b13-90ef-b70909f052fd admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:15.733118 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b628582-ea30-4b13-90ef-b70909f052fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:15.737954 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b628582-ea30-4b13-90ef-b70909f052fd admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:15.740176 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b628582-ea30-4b13-90ef-b70909f052fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:15.750534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b628582-ea30-4b13-90ef-b70909f052fd admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:15.875418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 874/1738] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:15 2022] POST /identity/v3/users => generated 291 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:15.880429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29c7ed-dda2-4f4f-a711-e1d4884963d0 service nova] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:15.917524 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e29c7ed-dda2-4f4f-a711-e1d4884963d0 service nova] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:15.965903 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700989d2-523c-4423-b196-10c7a4422e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:15.973116 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700989d2-523c-4423-b196-10c7a4422e1d admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:15.974031 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700989d2-523c-4423-b196-10c7a4422e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:15.976773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700989d2-523c-4423-b196-10c7a4422e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:15.980406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700989d2-523c-4423-b196-10c7a4422e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:15.990771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700989d2-523c-4423-b196-10c7a4422e1d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:16.045692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 865/1739] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:16.051967 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b628582-ea30-4b13-90ef-b70909f052fd admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:16.078795 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b628582-ea30-4b13-90ef-b70909f052fd admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:16.105264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564efaf-b8c8-4838-81d5-83afa751e6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:16.115260 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564efaf-b8c8-4838-81d5-83afa751e6d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:16.115260 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564efaf-b8c8-4838-81d5-83afa751e6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:16.115260 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564efaf-b8c8-4838-81d5-83afa751e6d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:16.115260 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564efaf-b8c8-4838-81d5-83afa751e6d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:16.115260 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564efaf-b8c8-4838-81d5-83afa751e6d5 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:16.174077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 875/1740] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:16 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:16.177039 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700989d2-523c-4423-b196-10c7a4422e1d admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:16.195346 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700989d2-523c-4423-b196-10c7a4422e1d admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:16.227889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a551b-682a-48b5-8dc8-152b693aebc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:16.232671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a551b-682a-48b5-8dc8-152b693aebc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:16.234833 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a551b-682a-48b5-8dc8-152b693aebc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:16.235784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9a551b-682a-48b5-8dc8-152b693aebc0 admin admin] PATH_INFO: `/v3/projects/4f75e29a9a864c12917bc5cb07a310e4/users/2a1e5c0d7ba84a299a822c96259e136c/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:16.349622 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a551b-682a-48b5-8dc8-152b693aebc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f75e29a9a864c12917bc5cb07a310e4, user_id=2a1e5c0d7ba84a299a822c96259e136c, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:16.377317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9a551b-682a-48b5-8dc8-152b693aebc0 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:16.437903 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 866/1741] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:16 2022] PUT /identity/v3/projects/4f75e29a9a864c12917bc5cb07a310e4/users/2a1e5c0d7ba84a299a822c96259e136c/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:16.449373 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a2e51-d4eb-489a-b10b-cbfc6cab832d None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:16.449787 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a2e51-d4eb-489a-b10b-cbfc6cab832d None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:16.450125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a2e51-d4eb-489a-b10b-cbfc6cab832d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:16.487133 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a551b-682a-48b5-8dc8-152b693aebc0 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:16.489061 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-1f9a551b-682a-48b5-8dc8-152b693aebc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:16.659467 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-296a2e51-d4eb-489a-b10b-cbfc6cab832d None None] MFA Rules not processed for user `2a1e5c0d7ba84a299a822c96259e136c`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:16.666623 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-296a2e51-d4eb-489a-b10b-cbfc6cab832d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:16.749964 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9a551b-682a-48b5-8dc8-152b693aebc0 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:16.771661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eac263-dff0-437c-9952-7033c0895a5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:16.777393 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac263-dff0-437c-9952-7033c0895a5a service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:16.777914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac263-dff0-437c-9952-7033c0895a5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:16.778426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac263-dff0-437c-9952-7033c0895a5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:16.805758 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-46eac263-dff0-437c-9952-7033c0895a5a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:16.890363 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:17.065592 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 876/1742] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 617 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:17.068670 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a2e51-d4eb-489a-b10b-cbfc6cab832d None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:17.075024 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-296a2e51-d4eb-489a-b10b-cbfc6cab832d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:17.101196 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eac263-dff0-437c-9952-7033c0895a5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:17.104019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eac263-dff0-437c-9952-7033c0895a5a service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:17.149759 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 867/1743] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 663 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:17.164134 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63d9fc-75c5-40bb-9f51-3b23a59e9dbf None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:17.165343 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63d9fc-75c5-40bb-9f51-3b23a59e9dbf None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:17.169336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe63d9fc-75c5-40bb-9f51-3b23a59e9dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:17.181248 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a2e51-d4eb-489a-b10b-cbfc6cab832d None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:17.186282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5eeb2d-b94e-4799-b046-f4805181a710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:17.187020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eeb2d-b94e-4799-b046-f4805181a710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:17.187365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eeb2d-b94e-4799-b046-f4805181a710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:17.187744 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5eeb2d-b94e-4799-b046-f4805181a710 admin admin] PATH_INFO: `/v3/users/4e689db6d2d6493c86fe408a5160033d` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:17.189393 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5eeb2d-b94e-4799-b046-f4805181a710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e689db6d2d6493c86fe408a5160033d)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:17.191443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5eeb2d-b94e-4799-b046-f4805181a710 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:17.262744 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-fe63d9fc-75c5-40bb-9f51-3b23a59e9dbf None None] MFA Rules not processed for user `6c2da427acb24c8cb6d667c6347858c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:17.266733 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-fe63d9fc-75c5-40bb-9f51-3b23a59e9dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:17.328976 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5d5eeb2d-b94e-4799-b046-f4805181a710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e689db6d2d6493c86fe408a5160033d'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:17.362583 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5d5eeb2d-b94e-4799-b046-f4805181a710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e689db6d2d6493c86fe408a5160033d'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:17.367277 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5d5eeb2d-b94e-4799-b046-f4805181a710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e689db6d2d6493c86fe408a5160033d'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:17.451347 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5d5eeb2d-b94e-4799-b046-f4805181a710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e689db6d2d6493c86fe408a5160033d'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:17.453249 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5d5eeb2d-b94e-4799-b046-f4805181a710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e689db6d2d6493c86fe408a5160033d'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:17.503298 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 868/1744] 10.222.0.36 () {58 vars in 1015 bytes} [Thu Aug 4 02:17:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:17.515190 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 877/1745] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:17 2022] DELETE /identity/v3/users/4e689db6d2d6493c86fe408a5160033d => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:17.520039 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9370aa-4c5a-45ca-b310-4ba82aa645ea None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:17.520426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63d9fc-75c5-40bb-9f51-3b23a59e9dbf None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:17.521202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9370aa-4c5a-45ca-b310-4ba82aa645ea None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:17.521651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9370aa-4c5a-45ca-b310-4ba82aa645ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:17.578433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe63d9fc-75c5-40bb-9f51-3b23a59e9dbf None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:17.629341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-8d9370aa-4c5a-45ca-b310-4ba82aa645ea None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:17.632233 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8d9370aa-4c5a-45ca-b310-4ba82aa645ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:17.641087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:17.778400 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef7095-02c5-4734-bd94-592e343d9b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:17.779067 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef7095-02c5-4734-bd94-592e343d9b85 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:17.779315 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef7095-02c5-4734-bd94-592e343d9b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:17.780918 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbef7095-02c5-4734-bd94-592e343d9b85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:17.781727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef7095-02c5-4734-bd94-592e343d9b85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:17.786524 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbef7095-02c5-4734-bd94-592e343d9b85 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:17.875233 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:17.915536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 878/1746] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:17.917333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9370aa-4c5a-45ca-b310-4ba82aa645ea None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:17.923873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8d9370aa-4c5a-45ca-b310-4ba82aa645ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:18.006079 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 869/1747] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:17 2022] POST /identity/v3/projects => generated 383 bytes in 502 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:18.007791 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef7095-02c5-4734-bd94-592e343d9b85 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:18.036937 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-fbef7095-02c5-4734-bd94-592e343d9b85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:18.207884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9370aa-4c5a-45ca-b310-4ba82aa645ea None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:18.294705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbef7095-02c5-4734-bd94-592e343d9b85 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:18.306572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43985e-1b6e-419e-9b7e-8fd598895b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:18.307747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43985e-1b6e-419e-9b7e-8fd598895b62 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:18.308237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43985e-1b6e-419e-9b7e-8fd598895b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:18.308799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a43985e-1b6e-419e-9b7e-8fd598895b62 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:18.312861 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43985e-1b6e-419e-9b7e-8fd598895b62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:18.316584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a43985e-1b6e-419e-9b7e-8fd598895b62 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:18.370076 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 870/1748] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:18 2022] POST /identity/v3/users => generated 298 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:18.372240 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43985e-1b6e-419e-9b7e-8fd598895b62 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:18.376137 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-1a43985e-1b6e-419e-9b7e-8fd598895b62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:18.448670 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f083528-b8a0-46fe-8521-1035a7f4f6f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:18.449244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f083528-b8a0-46fe-8521-1035a7f4f6f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:18.449635 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f083528-b8a0-46fe-8521-1035a7f4f6f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:18.449815 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f083528-b8a0-46fe-8521-1035a7f4f6f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:18.462517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-5f083528-b8a0-46fe-8521-1035a7f4f6f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:18.493038 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f083528-b8a0-46fe-8521-1035a7f4f6f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:18.494515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f083528-b8a0-46fe-8521-1035a7f4f6f6 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:18.535739 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 879/1749] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:18.539951 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f441a-d008-4d33-8c9b-44a076667e6b None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:18.540280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f441a-d008-4d33-8c9b-44a076667e6b None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:18.540558 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f441a-d008-4d33-8c9b-44a076667e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:18.561333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a43985e-1b6e-419e-9b7e-8fd598895b62 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:18.571167 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a47b7-8ae3-41cb-8e21-036405897a41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:18.574235 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a47b7-8ae3-41cb-8e21-036405897a41 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:18.575279 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a47b7-8ae3-41cb-8e21-036405897a41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:18.576606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a47b7-8ae3-41cb-8e21-036405897a41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:18.608145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a47b7-8ae3-41cb-8e21-036405897a41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:18.609965 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a47b7-8ae3-41cb-8e21-036405897a41 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:18.643958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 871/1750] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:18.645590 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a47b7-8ae3-41cb-8e21-036405897a41 service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:18.663871 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a47b7-8ae3-41cb-8e21-036405897a41 service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:18.669707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-a41f441a-d008-4d33-8c9b-44a076667e6b None None] MFA Rules not processed for user `26e73db77d93496883b66572d6bf35f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:18.669977 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaaaace-97b2-48ab-9e0f-af8361249970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:18.670688 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaaaace-97b2-48ab-9e0f-af8361249970 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:18.671078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaaaace-97b2-48ab-9e0f-af8361249970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:18.671423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaaaace-97b2-48ab-9e0f-af8361249970 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:18.672273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaaaace-97b2-48ab-9e0f-af8361249970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:18.674861 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaaaace-97b2-48ab-9e0f-af8361249970 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:18.679458 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a41f441a-d008-4d33-8c9b-44a076667e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:18.713730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 872/1751] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:18.716201 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaaaace-97b2-48ab-9e0f-af8361249970 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:18.742193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaaaace-97b2-48ab-9e0f-af8361249970 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:18.752646 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f9d7a-8eea-46a0-a1f4-e3ead4714b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:18.756265 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f9d7a-8eea-46a0-a1f4-e3ead4714b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:18.757185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f9d7a-8eea-46a0-a1f4-e3ead4714b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:18.757541 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f9d7a-8eea-46a0-a1f4-e3ead4714b33 admin admin] PATH_INFO: `/v3/projects/8388bf669b2244c396339ea11514e0be/users/2d468842d4194b919dd8a64cad1da583/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:18.797613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f9d7a-8eea-46a0-a1f4-e3ead4714b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8388bf669b2244c396339ea11514e0be, user_id=2d468842d4194b919dd8a64cad1da583, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:18.806093 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f9d7a-8eea-46a0-a1f4-e3ead4714b33 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:18.835447 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 873/1752] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:18 2022] PUT /identity/v3/projects/8388bf669b2244c396339ea11514e0be/users/2d468842d4194b919dd8a64cad1da583/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:18.839989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f9d7a-8eea-46a0-a1f4-e3ead4714b33 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:18.853512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:18.873571 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 880/1753] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:18.891536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f441a-d008-4d33-8c9b-44a076667e6b None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:18.897952 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a41f441a-d008-4d33-8c9b-44a076667e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:19.049383 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f9d7a-8eea-46a0-a1f4-e3ead4714b33 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:19.055859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:19.066313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de1b68-c7c9-4fd7-be5e-0db1d00210a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:19.067362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de1b68-c7c9-4fd7-be5e-0db1d00210a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:19.067799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de1b68-c7c9-4fd7-be5e-0db1d00210a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:19.068363 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39de1b68-c7c9-4fd7-be5e-0db1d00210a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:19.069346 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de1b68-c7c9-4fd7-be5e-0db1d00210a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:19.071155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39de1b68-c7c9-4fd7-be5e-0db1d00210a3 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:19.095914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 874/1754] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:19.097039 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de1b68-c7c9-4fd7-be5e-0db1d00210a3 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:19.145215 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39de1b68-c7c9-4fd7-be5e-0db1d00210a3 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:19.165632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3faeb-1b0d-4c75-96d9-7ecd7a761345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:19.168022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3faeb-1b0d-4c75-96d9-7ecd7a761345 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:19.169867 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3faeb-1b0d-4c75-96d9-7ecd7a761345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:19.170959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3faeb-1b0d-4c75-96d9-7ecd7a761345 admin admin] PATH_INFO: `/v3/projects/8388bf669b2244c396339ea11514e0be/users/2d468842d4194b919dd8a64cad1da583/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:19.186319 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3faeb-1b0d-4c75-96d9-7ecd7a761345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8388bf669b2244c396339ea11514e0be, user_id=2d468842d4194b919dd8a64cad1da583, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:19.187571 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f441a-d008-4d33-8c9b-44a076667e6b None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:19.195467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3faeb-1b0d-4c75-96d9-7ecd7a761345 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:19.241739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 875/1755] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:19 2022] PUT /identity/v3/projects/8388bf669b2244c396339ea11514e0be/users/2d468842d4194b919dd8a64cad1da583/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:19.252220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3bc4c1-edf6-44e4-b334-b9ad4a9cd654 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:19.252603 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3bc4c1-edf6-44e4-b334-b9ad4a9cd654 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:19.252975 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3bc4c1-edf6-44e4-b334-b9ad4a9cd654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:19.336194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-5c3bc4c1-edf6-44e4-b334-b9ad4a9cd654 None None] MFA Rules not processed for user `2d468842d4194b919dd8a64cad1da583`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:19.341185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-5c3bc4c1-edf6-44e4-b334-b9ad4a9cd654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:19.372971 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab815df2-9bff-44f3-9828-b5e525d90328 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:19.379108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab815df2-9bff-44f3-9828-b5e525d90328 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:19.379393 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab815df2-9bff-44f3-9828-b5e525d90328 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:19.379717 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab815df2-9bff-44f3-9828-b5e525d90328 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:19.545235 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 876/1756] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:19.558209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3bc4c1-edf6-44e4-b334-b9ad4a9cd654 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:19.559878 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-5c3bc4c1-edf6-44e4-b334-b9ad4a9cd654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:19.613834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab815df2-9bff-44f3-9828-b5e525d90328 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:19.618034 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab815df2-9bff-44f3-9828-b5e525d90328 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:19.666220 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 881/1757] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 776 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:17:19.723336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3bc4c1-edf6-44e4-b334-b9ad4a9cd654 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:19.728423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33880f60-79b6-4bf5-8590-bfc9a3c65491 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:19.729109 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33880f60-79b6-4bf5-8590-bfc9a3c65491 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:19.729440 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33880f60-79b6-4bf5-8590-bfc9a3c65491 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:19.729718 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33880f60-79b6-4bf5-8590-bfc9a3c65491 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:19.761567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33880f60-79b6-4bf5-8590-bfc9a3c65491 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:19.763031 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33880f60-79b6-4bf5-8590-bfc9a3c65491 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:19.813843 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 877/1758] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:19.983282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab815df2-9bff-44f3-9828-b5e525d90328 service nova] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:20.113044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab815df2-9bff-44f3-9828-b5e525d90328 service nova] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:20.117983 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ffa98-63ce-4fc2-a8f6-de75463f09a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:20.118785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ffa98-63ce-4fc2-a8f6-de75463f09a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:20.119292 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ffa98-63ce-4fc2-a8f6-de75463f09a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:20.119540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ffa98-63ce-4fc2-a8f6-de75463f09a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:20.121806 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-248ffa98-63ce-4fc2-a8f6-de75463f09a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:20.167336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33880f60-79b6-4bf5-8590-bfc9a3c65491 service cinder] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:20.169514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-33880f60-79b6-4bf5-8590-bfc9a3c65491 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:20.171956 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:20.203621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:20.233765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ffa98-63ce-4fc2-a8f6-de75463f09a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:20.235718 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ffa98-63ce-4fc2-a8f6-de75463f09a7 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:20.264261 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 882/1759] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:20.309528 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33880f60-79b6-4bf5-8590-bfc9a3c65491 service cinder] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:20.318183 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339be96-873d-410b-9f01-9877adee7d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:20.320187 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339be96-873d-410b-9f01-9877adee7d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:20.332681 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339be96-873d-410b-9f01-9877adee7d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:20.333562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339be96-873d-410b-9f01-9877adee7d97 admin admin] PATH_INFO: `/v3/projects/252324095c9b440dad83360a6b37b2cc` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:20.338055 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339be96-873d-410b-9f01-9877adee7d97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=252324095c9b440dad83360a6b37b2cc)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:20.341183 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339be96-873d-410b-9f01-9877adee7d97 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:20.462020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a339be96-873d-410b-9f01-9877adee7d97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '252324095c9b440dad83360a6b37b2cc', 'request_id': 'req-a339be96-873d-410b-9f01-9877adee7d97'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:20.476706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a339be96-873d-410b-9f01-9877adee7d97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '252324095c9b440dad83360a6b37b2cc', 'request_id': 'req-a339be96-873d-410b-9f01-9877adee7d97'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:20.491935 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a339be96-873d-410b-9f01-9877adee7d97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '252324095c9b440dad83360a6b37b2cc', 'request_id': 'req-a339be96-873d-410b-9f01-9877adee7d97'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:20.521485 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a339be96-873d-410b-9f01-9877adee7d97 admin admin] The token cache is being invalidate because project 252324095c9b440dad83360a6b37b2cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:20.521963 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a339be96-873d-410b-9f01-9877adee7d97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:20.523618 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a339be96-873d-410b-9f01-9877adee7d97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:20.526366 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 878/1760] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:20 2022] DELETE /identity/v3/projects/252324095c9b440dad83360a6b37b2cc => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:20.530503 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ffa98-63ce-4fc2-a8f6-de75463f09a7 service cinder] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:20.546337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ffa98-63ce-4fc2-a8f6-de75463f09a7 service cinder] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:20.769144 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808a842-d92c-408f-a728-e68424d3d0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:20.770735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808a842-d92c-408f-a728-e68424d3d0ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:20.770735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808a842-d92c-408f-a728-e68424d3d0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:20.770735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808a842-d92c-408f-a728-e68424d3d0ff admin admin] PATH_INFO: `/v3/users/fbb847bb36ea4e2fb5927669ff3a6cc7` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:20.771899 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808a842-d92c-408f-a728-e68424d3d0ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbb847bb36ea4e2fb5927669ff3a6cc7)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:20.788356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808a842-d92c-408f-a728-e68424d3d0ff admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:20.943830 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d808a842-d92c-408f-a728-e68424d3d0ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbb847bb36ea4e2fb5927669ff3a6cc7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:20.979152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d808a842-d92c-408f-a728-e68424d3d0ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbb847bb36ea4e2fb5927669ff3a6cc7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:20.988240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d808a842-d92c-408f-a728-e68424d3d0ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbb847bb36ea4e2fb5927669ff3a6cc7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:20.989775 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339be96-873d-410b-9f01-9877adee7d97 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:20.993909 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a339be96-873d-410b-9f01-9877adee7d97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:21.025316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339be96-873d-410b-9f01-9877adee7d97 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:21.040372 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb3c76-760b-47a4-87a0-05877d1b52a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:21.041198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb3c76-760b-47a4-87a0-05877d1b52a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:21.041667 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb3c76-760b-47a4-87a0-05877d1b52a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:21.041948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb3c76-760b-47a4-87a0-05877d1b52a2 admin admin] PATH_INFO: `/v3/users/2a1e5c0d7ba84a299a822c96259e136c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:21.046607 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb3c76-760b-47a4-87a0-05877d1b52a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a1e5c0d7ba84a299a822c96259e136c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:21.054137 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d808a842-d92c-408f-a728-e68424d3d0ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbb847bb36ea4e2fb5927669ff3a6cc7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:21.062988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d808a842-d92c-408f-a728-e68424d3d0ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbb847bb36ea4e2fb5927669ff3a6cc7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:21.063436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb3c76-760b-47a4-87a0-05877d1b52a2 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:21.086119 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 883/1761] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:20 2022] DELETE /identity/v3/users/fbb847bb36ea4e2fb5927669ff3a6cc7 => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:21.219578 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-53bb3c76-760b-47a4-87a0-05877d1b52a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a1e5c0d7ba84a299a822c96259e136c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:21.220311 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-53bb3c76-760b-47a4-87a0-05877d1b52a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a1e5c0d7ba84a299a822c96259e136c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:21.221025 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-53bb3c76-760b-47a4-87a0-05877d1b52a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a1e5c0d7ba84a299a822c96259e136c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:21.236340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-53bb3c76-760b-47a4-87a0-05877d1b52a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a1e5c0d7ba84a299a822c96259e136c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:21.246026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-53bb3c76-760b-47a4-87a0-05877d1b52a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a1e5c0d7ba84a299a822c96259e136c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:21.271877 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 879/1762] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:20 2022] DELETE /identity/v3/users/2a1e5c0d7ba84a299a822c96259e136c => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:21.275305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1888e6-990e-4ac4-bf1f-20680ab67edf None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:21.275587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1888e6-990e-4ac4-bf1f-20680ab67edf None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:21.275861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1888e6-990e-4ac4-bf1f-20680ab67edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:21.334472 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-cf1888e6-990e-4ac4-bf1f-20680ab67edf None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:21.342744 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-cf1888e6-990e-4ac4-bf1f-20680ab67edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:21.506205 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:21.556196 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 884/1763] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:21.726458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb3c76-760b-47a4-87a0-05877d1b52a2 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:21.727929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-53bb3c76-760b-47a4-87a0-05877d1b52a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:21.783634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:22.012933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb3c76-760b-47a4-87a0-05877d1b52a2 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:22.031028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44a3812-976f-4375-baeb-c6eec94121c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:22.031906 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a3812-976f-4375-baeb-c6eec94121c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:22.032231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a3812-976f-4375-baeb-c6eec94121c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:22.032466 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a3812-976f-4375-baeb-c6eec94121c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:22.073211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a3812-976f-4375-baeb-c6eec94121c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:22.074924 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44a3812-976f-4375-baeb-c6eec94121c8 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:22.103136 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 880/1764] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:23.108152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881498b2-5e51-46b5-87a8-21fee6aa8219 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:23.109345 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881498b2-5e51-46b5-87a8-21fee6aa8219 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:23.110615 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881498b2-5e51-46b5-87a8-21fee6aa8219 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:23.113291 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 885/1765] 10.222.0.45 () {54 vars in 921 bytes} [Thu Aug 4 02:17:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:17:23.402985 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9ef74-25bb-48fb-b42e-878bec8da150 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:23.403852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9ef74-25bb-48fb-b42e-878bec8da150 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:23.405169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9ef74-25bb-48fb-b42e-878bec8da150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:23.506228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-2ef9ef74-25bb-48fb-b42e-878bec8da150 None None] MFA Rules not processed for user `c571406d2a3f40dab035a918399d6f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:23.508017 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2ef9ef74-25bb-48fb-b42e-878bec8da150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:23.907862 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 881/1766] 10.222.0.45 () {58 vars in 1018 bytes} [Thu Aug 4 02:17:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 509 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:24.189960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881498b2-5e51-46b5-87a8-21fee6aa8219 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:24.191910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-881498b2-5e51-46b5-87a8-21fee6aa8219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:24.246171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881498b2-5e51-46b5-87a8-21fee6aa8219 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:24.284382 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef4613-5ae3-4f34-aea1-a883ce71ea21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:24.286944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef4613-5ae3-4f34-aea1-a883ce71ea21 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:24.287684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef4613-5ae3-4f34-aea1-a883ce71ea21 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:24.290786 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef4613-5ae3-4f34-aea1-a883ce71ea21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:24.319873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ef4613-5ae3-4f34-aea1-a883ce71ea21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:24.327590 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ef4613-5ae3-4f34-aea1-a883ce71ea21 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:24.374730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 886/1767] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:25.219463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9ef74-25bb-48fb-b42e-878bec8da150 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:25.221375 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2ef9ef74-25bb-48fb-b42e-878bec8da150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:25.262988 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9ef74-25bb-48fb-b42e-878bec8da150 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:25.279914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b7494-b556-4016-a0ee-a3e92321232a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:25.284068 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b7494-b556-4016-a0ee-a3e92321232a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:25.288590 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b7494-b556-4016-a0ee-a3e92321232a admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:25.289210 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b7494-b556-4016-a0ee-a3e92321232a admin admin] PATH_INFO: `/v3/projects/2842783593654022b700982a818b3df2` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:25.294082 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b7494-b556-4016-a0ee-a3e92321232a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2842783593654022b700982a818b3df2)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:25.298262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b7494-b556-4016-a0ee-a3e92321232a admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:25.445162 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0a6b7494-b556-4016-a0ee-a3e92321232a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2842783593654022b700982a818b3df2', 'request_id': 'req-0a6b7494-b556-4016-a0ee-a3e92321232a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:25.459187 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0a6b7494-b556-4016-a0ee-a3e92321232a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2842783593654022b700982a818b3df2', 'request_id': 'req-0a6b7494-b556-4016-a0ee-a3e92321232a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:25.488474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0a6b7494-b556-4016-a0ee-a3e92321232a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2842783593654022b700982a818b3df2', 'request_id': 'req-0a6b7494-b556-4016-a0ee-a3e92321232a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:25.505624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0a6b7494-b556-4016-a0ee-a3e92321232a admin admin] The token cache is being invalidate because project 2842783593654022b700982a818b3df2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:25.506146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0a6b7494-b556-4016-a0ee-a3e92321232a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:25.507631 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0a6b7494-b556-4016-a0ee-a3e92321232a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:25.514469 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 882/1768] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:25 2022] DELETE /identity/v3/projects/2842783593654022b700982a818b3df2 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:25.515813 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e38ea-6d7d-41b8-af23-7565eb3e0bae None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:25.516383 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e38ea-6d7d-41b8-af23-7565eb3e0bae None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:25.516383 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e38ea-6d7d-41b8-af23-7565eb3e0bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:25.565028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-421e38ea-6d7d-41b8-af23-7565eb3e0bae None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:25.566505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-421e38ea-6d7d-41b8-af23-7565eb3e0bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:25.788565 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 887/1769] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:25.797003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec0da9-e631-4625-b7b4-69422f0045a3 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:25.797292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec0da9-e631-4625-b7b4-69422f0045a3 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:25.797575 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec0da9-e631-4625-b7b4-69422f0045a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:25.849273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-aaec0da9-e631-4625-b7b4-69422f0045a3 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:25.851134 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-aaec0da9-e631-4625-b7b4-69422f0045a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:25.890015 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d053c-0933-434c-8851-4af02747128c None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:25.890422 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 883/1770] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:25.890577 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d053c-0933-434c-8851-4af02747128c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:25.890919 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d053c-0933-434c-8851-4af02747128c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:25.891387 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec0da9-e631-4625-b7b4-69422f0045a3 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:25.910756 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec0da9-e631-4625-b7b4-69422f0045a3 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:25.924167 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a3723-d76e-4be1-8646-3ae129850a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:25.925102 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a3723-d76e-4be1-8646-3ae129850a48 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:25.925102 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a3723-d76e-4be1-8646-3ae129850a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:25.925387 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a3723-d76e-4be1-8646-3ae129850a48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:25.926096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a3723-d76e-4be1-8646-3ae129850a48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:17:25.926408 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a3723-d76e-4be1-8646-3ae129850a48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:25.928154 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a3723-d76e-4be1-8646-3ae129850a48 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:25.935807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-389d053c-0933-434c-8851-4af02747128c None None] MFA Rules not processed for user `df5d3de1afa945c3b0873050eca06c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:25.937387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-389d053c-0933-434c-8851-4af02747128c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:25.957048 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 884/1771] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:17:25.958153 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a3723-d76e-4be1-8646-3ae129850a48 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:25.982735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a3723-d76e-4be1-8646-3ae129850a48 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:25.997417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054ac99-475d-45ab-b7d5-8725bba21057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:25.998087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054ac99-475d-45ab-b7d5-8725bba21057 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:25.998660 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054ac99-475d-45ab-b7d5-8725bba21057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:25.999720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054ac99-475d-45ab-b7d5-8725bba21057 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:25.999720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054ac99-475d-45ab-b7d5-8725bba21057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:26.002125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054ac99-475d-45ab-b7d5-8725bba21057 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:26.028854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 888/1772] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:26.054837 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:26.083225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc85453-b528-433f-99a5-1a79f2403690 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:26.083407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 885/1773] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:25 2022] POST /identity/v3/projects => generated 355 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:26.084429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc85453-b528-433f-99a5-1a79f2403690 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:26.086576 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc85453-b528-433f-99a5-1a79f2403690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:26.088412 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054ac99-475d-45ab-b7d5-8725bba21057 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:26.175115 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-8bc85453-b528-433f-99a5-1a79f2403690 None None] MFA Rules not processed for user `76703dc82dc644ef8d32f7e259ddea6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:26.180888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8bc85453-b528-433f-99a5-1a79f2403690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:26.214239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054ac99-475d-45ab-b7d5-8725bba21057 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:26.225119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dabcb4-cbdd-413c-984d-dfb9cfa00567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:26.225739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dabcb4-cbdd-413c-984d-dfb9cfa00567 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:26.226013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dabcb4-cbdd-413c-984d-dfb9cfa00567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:26.226289 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dabcb4-cbdd-413c-984d-dfb9cfa00567 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:26.226965 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dabcb4-cbdd-413c-984d-dfb9cfa00567 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:26.229589 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dabcb4-cbdd-413c-984d-dfb9cfa00567 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:26.279295 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 886/1774] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:26 2022] POST /identity/v3/users => generated 278 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:26.283898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dabcb4-cbdd-413c-984d-dfb9cfa00567 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:26.301551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dabcb4-cbdd-413c-984d-dfb9cfa00567 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:26.301972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 889/1775] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:26.307019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893b1cb-496f-46cd-a2b2-3062b6e843cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:26.307672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893b1cb-496f-46cd-a2b2-3062b6e843cc admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:26.307943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893b1cb-496f-46cd-a2b2-3062b6e843cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:26.308224 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893b1cb-496f-46cd-a2b2-3062b6e843cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:26.308962 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893b1cb-496f-46cd-a2b2-3062b6e843cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:26.310602 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3893b1cb-496f-46cd-a2b2-3062b6e843cc admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:26.327779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc85453-b528-433f-99a5-1a79f2403690 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:26.332531 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 887/1776] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:26.349436 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc85453-b528-433f-99a5-1a79f2403690 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:26.364590 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec7997-082d-4249-ac3f-b45b1d8195e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:26.365277 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec7997-082d-4249-ac3f-b45b1d8195e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:26.365618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec7997-082d-4249-ac3f-b45b1d8195e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:26.366125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec7997-082d-4249-ac3f-b45b1d8195e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:26.367031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec7997-082d-4249-ac3f-b45b1d8195e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:26.368946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec7997-082d-4249-ac3f-b45b1d8195e7 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:26.394321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 890/1777] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:26.400841 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893b1cb-496f-46cd-a2b2-3062b6e843cc admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:26.416636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3893b1cb-496f-46cd-a2b2-3062b6e843cc admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:26.431993 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec7997-082d-4249-ac3f-b45b1d8195e7 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:26.435925 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381afcf-c264-4815-9853-25789af9a338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:26.436650 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381afcf-c264-4815-9853-25789af9a338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:26.436942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381afcf-c264-4815-9853-25789af9a338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:26.437204 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381afcf-c264-4815-9853-25789af9a338 admin admin] PATH_INFO: `/v3/projects/e5ee77f9c0594d7990feee496314e3f9/users/70337b7913654cfdbb929ca4457aef21/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:26.438713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-bbec7997-082d-4249-ac3f-b45b1d8195e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:26.467727 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec7997-082d-4249-ac3f-b45b1d8195e7 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:26.479668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381afcf-c264-4815-9853-25789af9a338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ee77f9c0594d7990feee496314e3f9, user_id=70337b7913654cfdbb929ca4457aef21, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:26.482423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381afcf-c264-4815-9853-25789af9a338 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:26.485649 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100d20d-e64c-4316-b3c7-139b778b2dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:26.486248 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100d20d-e64c-4316-b3c7-139b778b2dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:26.486528 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100d20d-e64c-4316-b3c7-139b778b2dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:26.486798 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100d20d-e64c-4316-b3c7-139b778b2dcc admin admin] PATH_INFO: `/v3/projects/4f75e29a9a864c12917bc5cb07a310e4` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:26.488905 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100d20d-e64c-4316-b3c7-139b778b2dcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f75e29a9a864c12917bc5cb07a310e4)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:26.494028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b100d20d-e64c-4316-b3c7-139b778b2dcc admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:26.521161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 888/1778] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:26 2022] PUT /identity/v3/projects/e5ee77f9c0594d7990feee496314e3f9/users/70337b7913654cfdbb929ca4457aef21/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:26.523630 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6b8b2-c1c5-4bd4-b4f1-1d56927faf0c None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:26.523909 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6b8b2-c1c5-4bd4-b4f1-1d56927faf0c None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:26.524180 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6b8b2-c1c5-4bd4-b4f1-1d56927faf0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:26.528463 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:26.581852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-95d6b8b2-c1c5-4bd4-b4f1-1d56927faf0c None None] MFA Rules not processed for user `70337b7913654cfdbb929ca4457aef21`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:26.583073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-95d6b8b2-c1c5-4bd4-b4f1-1d56927faf0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:26.592965 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b100d20d-e64c-4316-b3c7-139b778b2dcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f75e29a9a864c12917bc5cb07a310e4', 'request_id': 'req-b100d20d-e64c-4316-b3c7-139b778b2dcc'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:26.607817 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b100d20d-e64c-4316-b3c7-139b778b2dcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f75e29a9a864c12917bc5cb07a310e4', 'request_id': 'req-b100d20d-e64c-4316-b3c7-139b778b2dcc'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:26.614920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b100d20d-e64c-4316-b3c7-139b778b2dcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f75e29a9a864c12917bc5cb07a310e4', 'request_id': 'req-b100d20d-e64c-4316-b3c7-139b778b2dcc'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:26.643353 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b100d20d-e64c-4316-b3c7-139b778b2dcc admin admin] The token cache is being invalidate because project 4f75e29a9a864c12917bc5cb07a310e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:26.643635 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b100d20d-e64c-4316-b3c7-139b778b2dcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:26.645864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b100d20d-e64c-4316-b3c7-139b778b2dcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:26.650288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 891/1779] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:26 2022] DELETE /identity/v3/projects/4f75e29a9a864c12917bc5cb07a310e4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:26.654394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100d20d-e64c-4316-b3c7-139b778b2dcc admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:26.657648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b100d20d-e64c-4316-b3c7-139b778b2dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:26.745071 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 889/1780] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:26 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:26.746631 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6b8b2-c1c5-4bd4-b4f1-1d56927faf0c None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:26.748573 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-95d6b8b2-c1c5-4bd4-b4f1-1d56927faf0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:26.799267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100d20d-e64c-4316-b3c7-139b778b2dcc admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:26.814544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e486a-4eff-47ae-9b14-41b4997fdfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:26.815930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e486a-4eff-47ae-9b14-41b4997fdfae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:26.816496 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e486a-4eff-47ae-9b14-41b4997fdfae admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:26.817046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e486a-4eff-47ae-9b14-41b4997fdfae admin admin] PATH_INFO: `/v3/users/2d468842d4194b919dd8a64cad1da583` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:26.820328 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e486a-4eff-47ae-9b14-41b4997fdfae admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d468842d4194b919dd8a64cad1da583)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:26.822894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e486a-4eff-47ae-9b14-41b4997fdfae admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:26.844376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:26.900102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a45e486a-4eff-47ae-9b14-41b4997fdfae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d468842d4194b919dd8a64cad1da583'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:26.903312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6b8b2-c1c5-4bd4-b4f1-1d56927faf0c None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:26.911091 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a45e486a-4eff-47ae-9b14-41b4997fdfae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d468842d4194b919dd8a64cad1da583'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:26.913817 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a45e486a-4eff-47ae-9b14-41b4997fdfae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d468842d4194b919dd8a64cad1da583'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:26.919045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c8f53-97c2-481d-8741-0ae17fc4925b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:26.919697 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c8f53-97c2-481d-8741-0ae17fc4925b admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:26.919979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c8f53-97c2-481d-8741-0ae17fc4925b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:26.920255 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c8f53-97c2-481d-8741-0ae17fc4925b admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:26.921165 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c8f53-97c2-481d-8741-0ae17fc4925b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:26.922881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1c8f53-97c2-481d-8741-0ae17fc4925b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:26.929274 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a45e486a-4eff-47ae-9b14-41b4997fdfae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d468842d4194b919dd8a64cad1da583'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:26.930326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a45e486a-4eff-47ae-9b14-41b4997fdfae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d468842d4194b919dd8a64cad1da583'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:26.957718 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 892/1781] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:26 2022] DELETE /identity/v3/users/2d468842d4194b919dd8a64cad1da583 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:26.985994 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 890/1782] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:26 2022] POST /identity/v3/projects => generated 353 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:17:26.987902 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e486a-4eff-47ae-9b14-41b4997fdfae admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:26.990231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a45e486a-4eff-47ae-9b14-41b4997fdfae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:27.030228 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:27.096657 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e486a-4eff-47ae-9b14-41b4997fdfae admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:27.230415 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2b5b5-248c-4d6c-80f9-1fc6a0e3833f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:27.231685 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2b5b5-248c-4d6c-80f9-1fc6a0e3833f admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:27.232266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2b5b5-248c-4d6c-80f9-1fc6a0e3833f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:27.232779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2b5b5-248c-4d6c-80f9-1fc6a0e3833f admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:27.234342 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2b5b5-248c-4d6c-80f9-1fc6a0e3833f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:27.237082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2b5b5-248c-4d6c-80f9-1fc6a0e3833f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:27.264237 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 893/1783] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:26 2022] POST /identity/v3/users => generated 283 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:27.269244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c8f53-97c2-481d-8741-0ae17fc4925b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:27.286710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1c8f53-97c2-481d-8741-0ae17fc4925b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:27.293056 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14313c4b-0d1c-461a-9e80-1ded396a5685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:27.294135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14313c4b-0d1c-461a-9e80-1ded396a5685 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:27.294663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14313c4b-0d1c-461a-9e80-1ded396a5685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:27.295151 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14313c4b-0d1c-461a-9e80-1ded396a5685 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:27.296593 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14313c4b-0d1c-461a-9e80-1ded396a5685 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:27.300392 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14313c4b-0d1c-461a-9e80-1ded396a5685 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:27.319975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a28727-b927-49eb-947c-7ce0b440db02 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:27.320323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a28727-b927-49eb-947c-7ce0b440db02 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:27.320607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a28727-b927-49eb-947c-7ce0b440db02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:27.321192 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 891/1784] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:27.332263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14313c4b-0d1c-461a-9e80-1ded396a5685 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:27.351907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14313c4b-0d1c-461a-9e80-1ded396a5685 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:27.360313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36738161-6164-4bd7-8482-6563e45108b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:27.361403 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36738161-6164-4bd7-8482-6563e45108b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:27.363842 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36738161-6164-4bd7-8482-6563e45108b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:27.364503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36738161-6164-4bd7-8482-6563e45108b6 admin admin] PATH_INFO: `/v3/projects/d72cb0368e5a4522975f61b451fb0aad/users/02e8736d24cb44f3af9631ee40bed495/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:27.392512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-48a28727-b927-49eb-947c-7ce0b440db02 None None] MFA Rules not processed for user `26e73db77d93496883b66572d6bf35f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:27.399108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36738161-6164-4bd7-8482-6563e45108b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d72cb0368e5a4522975f61b451fb0aad, user_id=02e8736d24cb44f3af9631ee40bed495, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:27.401034 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-48a28727-b927-49eb-947c-7ce0b440db02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:27.402823 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36738161-6164-4bd7-8482-6563e45108b6 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:27.442004 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 892/1785] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:27 2022] PUT /identity/v3/projects/d72cb0368e5a4522975f61b451fb0aad/users/02e8736d24cb44f3af9631ee40bed495/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:27.443937 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36738161-6164-4bd7-8482-6563e45108b6 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:27.555551 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 894/1786] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:27.661000 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36738161-6164-4bd7-8482-6563e45108b6 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:27.681609 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22f9ae-0553-419e-a4b6-055bbd8f4d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:27.682818 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22f9ae-0553-419e-a4b6-055bbd8f4d52 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:27.683383 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22f9ae-0553-419e-a4b6-055bbd8f4d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:27.683881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22f9ae-0553-419e-a4b6-055bbd8f4d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:27.685431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22f9ae-0553-419e-a4b6-055bbd8f4d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:27.696409 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22f9ae-0553-419e-a4b6-055bbd8f4d52 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:27.730539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 893/1787] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:27 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:27.738502 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a28727-b927-49eb-947c-7ce0b440db02 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:27.753233 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a28727-b927-49eb-947c-7ce0b440db02 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:27.766660 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85ea99-1dc7-44fd-bbb7-5d2e80e51d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:27.766660 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85ea99-1dc7-44fd-bbb7-5d2e80e51d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:27.766939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85ea99-1dc7-44fd-bbb7-5d2e80e51d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:27.769162 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85ea99-1dc7-44fd-bbb7-5d2e80e51d6b admin admin] PATH_INFO: `/v3/projects/d72cb0368e5a4522975f61b451fb0aad/users/02e8736d24cb44f3af9631ee40bed495/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:27.791409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb85ea99-1dc7-44fd-bbb7-5d2e80e51d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d72cb0368e5a4522975f61b451fb0aad, user_id=02e8736d24cb44f3af9631ee40bed495, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:27.799492 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb85ea99-1dc7-44fd-bbb7-5d2e80e51d6b admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:27.867516 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 895/1788] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:27 2022] PUT /identity/v3/projects/d72cb0368e5a4522975f61b451fb0aad/users/02e8736d24cb44f3af9631ee40bed495/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:27.869887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679150c-a937-46b5-b4fa-adda4ca9fec6 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:27.870253 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679150c-a937-46b5-b4fa-adda4ca9fec6 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:27.870625 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679150c-a937-46b5-b4fa-adda4ca9fec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:27.961233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-7679150c-a937-46b5-b4fa-adda4ca9fec6 None None] MFA Rules not processed for user `02e8736d24cb44f3af9631ee40bed495`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:27.962677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7679150c-a937-46b5-b4fa-adda4ca9fec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:27.983189 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85ea99-1dc7-44fd-bbb7-5d2e80e51d6b admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:27.986831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-bb85ea99-1dc7-44fd-bbb7-5d2e80e51d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:28.137828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:28.239964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85ea99-1dc7-44fd-bbb7-5d2e80e51d6b admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:28.246249 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 894/1789] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:27 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:28.253973 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2eb89-c99d-48a7-82ef-607b2b258590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:28.254740 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2eb89-c99d-48a7-82ef-607b2b258590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:28.255123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2eb89-c99d-48a7-82ef-607b2b258590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:28.255576 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2eb89-c99d-48a7-82ef-607b2b258590 admin admin] PATH_INFO: `/v3/projects/8388bf669b2244c396339ea11514e0be` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:28.258049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2eb89-c99d-48a7-82ef-607b2b258590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8388bf669b2244c396339ea11514e0be)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:28.260854 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679150c-a937-46b5-b4fa-adda4ca9fec6 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:28.262928 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2eb89-c99d-48a7-82ef-607b2b258590 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:28.263748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7679150c-a937-46b5-b4fa-adda4ca9fec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:28.277174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:28.446267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679150c-a937-46b5-b4fa-adda4ca9fec6 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:28.464489 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-84e2eb89-c99d-48a7-82ef-607b2b258590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8388bf669b2244c396339ea11514e0be', 'request_id': 'req-84e2eb89-c99d-48a7-82ef-607b2b258590'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:28.487735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-84e2eb89-c99d-48a7-82ef-607b2b258590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8388bf669b2244c396339ea11514e0be', 'request_id': 'req-84e2eb89-c99d-48a7-82ef-607b2b258590'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:28.524799 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-84e2eb89-c99d-48a7-82ef-607b2b258590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8388bf669b2244c396339ea11514e0be', 'request_id': 'req-84e2eb89-c99d-48a7-82ef-607b2b258590'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:28.579117 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-84e2eb89-c99d-48a7-82ef-607b2b258590 admin admin] The token cache is being invalidate because project 8388bf669b2244c396339ea11514e0be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:28.581997 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-84e2eb89-c99d-48a7-82ef-607b2b258590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:28.581997 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-84e2eb89-c99d-48a7-82ef-607b2b258590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:28.596030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 896/1790] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:27 2022] DELETE /identity/v3/projects/8388bf669b2244c396339ea11514e0be => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:28.601903 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f36ea-6f0d-4f84-82a9-93ac0009a489 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:28.602557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f36ea-6f0d-4f84-82a9-93ac0009a489 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:28.602843 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f36ea-6f0d-4f84-82a9-93ac0009a489 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:28.603315 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f36ea-6f0d-4f84-82a9-93ac0009a489 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:28.604464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec194f5-dabc-464a-9930-d5bca764f501 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:28.606084 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec194f5-dabc-464a-9930-d5bca764f501 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:28.606902 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec194f5-dabc-464a-9930-d5bca764f501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:28.609668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-094f36ea-6f0d-4f84-82a9-93ac0009a489 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:28.705889 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-dec194f5-dabc-464a-9930-d5bca764f501 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:28.713853 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-dec194f5-dabc-464a-9930-d5bca764f501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:28.805009 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094f36ea-6f0d-4f84-82a9-93ac0009a489 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:28.810473 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094f36ea-6f0d-4f84-82a9-93ac0009a489 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:28.845888 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 895/1791] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:28 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:28.994649 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 897/1792] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:28.994944 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d13af1-ec50-454b-86c6-7173b1d6b184 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:28.995671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d13af1-ec50-454b-86c6-7173b1d6b184 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:28.995671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d13af1-ec50-454b-86c6-7173b1d6b184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:29.082071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec194f5-dabc-464a-9930-d5bca764f501 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:29.083599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-dec194f5-dabc-464a-9930-d5bca764f501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:29.088271 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-48d13af1-ec50-454b-86c6-7173b1d6b184 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:29.099012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-48d13af1-ec50-454b-86c6-7173b1d6b184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:29.118944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec194f5-dabc-464a-9930-d5bca764f501 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:29.132161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 896/1793] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:29.148499 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b482afa-2083-491a-b0ac-4e15484877c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:29.149205 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b482afa-2083-491a-b0ac-4e15484877c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:29.149488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b482afa-2083-491a-b0ac-4e15484877c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:29.149771 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b482afa-2083-491a-b0ac-4e15484877c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:29.151703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9b482afa-2083-491a-b0ac-4e15484877c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:29.154122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d13af1-ec50-454b-86c6-7173b1d6b184 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:29.182448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d13af1-ec50-454b-86c6-7173b1d6b184 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:29.197402 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d47552-149f-470e-900a-d4c12ede95f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:29.204920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d47552-149f-470e-900a-d4c12ede95f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:29.206793 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d47552-149f-470e-900a-d4c12ede95f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:29.207309 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d47552-149f-470e-900a-d4c12ede95f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:29.210944 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d47552-149f-470e-900a-d4c12ede95f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:17:29.211354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d47552-149f-470e-900a-d4c12ede95f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:29.213291 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d47552-149f-470e-900a-d4c12ede95f5 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:29.283380 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 897/1794] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:17:29.287281 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d47552-149f-470e-900a-d4c12ede95f5 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:29.294633 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b482afa-2083-491a-b0ac-4e15484877c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:29.296835 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b482afa-2083-491a-b0ac-4e15484877c5 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:29.322804 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d47552-149f-470e-900a-d4c12ede95f5 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:29.326017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 898/1795] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:17:29.339060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc3392-aca0-46ac-bb3e-9d0532b2cefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:29.340591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc3392-aca0-46ac-bb3e-9d0532b2cefb admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:29.341725 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc3392-aca0-46ac-bb3e-9d0532b2cefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:29.342245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc3392-aca0-46ac-bb3e-9d0532b2cefb admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:29.354203 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc3392-aca0-46ac-bb3e-9d0532b2cefb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:29.362578 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc3392-aca0-46ac-bb3e-9d0532b2cefb admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:29.510050 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b482afa-2083-491a-b0ac-4e15484877c5 service cinder] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:29.513992 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 898/1796] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:29 2022] POST /identity/v3/projects => generated 383 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:29.530611 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b482afa-2083-491a-b0ac-4e15484877c5 service cinder] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:29.587678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:29.792344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fee214-8e64-4811-8427-90ebe9305032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:29.793092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fee214-8e64-4811-8427-90ebe9305032 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:29.793375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fee214-8e64-4811-8427-90ebe9305032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:29.794033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fee214-8e64-4811-8427-90ebe9305032 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:29.794796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fee214-8e64-4811-8427-90ebe9305032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:29.798013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fee214-8e64-4811-8427-90ebe9305032 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:29.849837 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 899/1797] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:29 2022] POST /identity/v3/users => generated 292 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:29.852725 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc3392-aca0-46ac-bb3e-9d0532b2cefb admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:29.885614 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc3392-aca0-46ac-bb3e-9d0532b2cefb admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:29.905078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200d6d3-d09c-43b2-977a-c9a7251d90ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:29.905901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200d6d3-d09c-43b2-977a-c9a7251d90ba admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:29.906318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200d6d3-d09c-43b2-977a-c9a7251d90ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:29.906731 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c200d6d3-d09c-43b2-977a-c9a7251d90ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:29.911226 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200d6d3-d09c-43b2-977a-c9a7251d90ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:29.915665 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c200d6d3-d09c-43b2-977a-c9a7251d90ba admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:29.943787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 899/1798] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:29.945216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fee214-8e64-4811-8427-90ebe9305032 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:29.962918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fee214-8e64-4811-8427-90ebe9305032 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:29.975074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b9e9a-793d-4206-af17-655609b657a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:29.976336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b9e9a-793d-4206-af17-655609b657a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:29.976723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b9e9a-793d-4206-af17-655609b657a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:29.977044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b9e9a-793d-4206-af17-655609b657a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:29.990558 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b9e9a-793d-4206-af17-655609b657a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:30.005894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63b9e9a-793d-4206-af17-655609b657a2 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:30.066345 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 900/1799] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:30.067696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200d6d3-d09c-43b2-977a-c9a7251d90ba admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:30.088147 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c200d6d3-d09c-43b2-977a-c9a7251d90ba admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:30.109286 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c92a7f-6b94-417a-b832-2a417c7ae952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:30.110464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c92a7f-6b94-417a-b832-2a417c7ae952 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:30.110823 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c92a7f-6b94-417a-b832-2a417c7ae952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:30.111099 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c92a7f-6b94-417a-b832-2a417c7ae952 admin admin] PATH_INFO: `/v3/projects/a850f1723cd645ec95113b4449a32479/users/4d0b9d0be26c46d78aa962c49f8dd718/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:30.176207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c92a7f-6b94-417a-b832-2a417c7ae952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a850f1723cd645ec95113b4449a32479, user_id=4d0b9d0be26c46d78aa962c49f8dd718, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:30.179008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c92a7f-6b94-417a-b832-2a417c7ae952 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:30.210984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:30.219297 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 900/1800] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:30 2022] PUT /identity/v3/projects/a850f1723cd645ec95113b4449a32479/users/4d0b9d0be26c46d78aa962c49f8dd718/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:30.232155 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93092c83-18bb-448b-b5a4-6d7be70e87fa None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:30.232517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93092c83-18bb-448b-b5a4-6d7be70e87fa None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:30.232737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93092c83-18bb-448b-b5a4-6d7be70e87fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:30.367309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-93092c83-18bb-448b-b5a4-6d7be70e87fa None None] MFA Rules not processed for user `4d0b9d0be26c46d78aa962c49f8dd718`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:30.370732 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-93092c83-18bb-448b-b5a4-6d7be70e87fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:30.731367 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 901/1801] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 500 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:30.732005 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c92a7f-6b94-417a-b832-2a417c7ae952 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:31.037071 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c92a7f-6b94-417a-b832-2a417c7ae952 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:31.055010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca1e00-61d5-470a-9115-986eb2b5ae97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:31.055693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca1e00-61d5-470a-9115-986eb2b5ae97 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:31.055997 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca1e00-61d5-470a-9115-986eb2b5ae97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:31.056293 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca1e00-61d5-470a-9115-986eb2b5ae97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:31.057099 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca1e00-61d5-470a-9115-986eb2b5ae97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:31.075580 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca1e00-61d5-470a-9115-986eb2b5ae97 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:31.164557 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93092c83-18bb-448b-b5a4-6d7be70e87fa None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:31.176832 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 901/1802] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:30 2022] POST /identity/v3/projects => generated 385 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:31.191565 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93092c83-18bb-448b-b5a4-6d7be70e87fa None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:31.317861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4166f-5f2f-4689-95cb-0f4f776f5ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:31.318716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4166f-5f2f-4689-95cb-0f4f776f5ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:31.319662 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4166f-5f2f-4689-95cb-0f4f776f5ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:31.319937 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4166f-5f2f-4689-95cb-0f4f776f5ef3 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:31.323655 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4166f-5f2f-4689-95cb-0f4f776f5ef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:31.325718 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4166f-5f2f-4689-95cb-0f4f776f5ef3 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:31.358238 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 902/1803] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:31 2022] POST /identity/v3/users => generated 299 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:17:31.362445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca1e00-61d5-470a-9115-986eb2b5ae97 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:31.375826 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca1e00-61d5-470a-9115-986eb2b5ae97 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:31.382638 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4a95a-f74f-4784-804a-85c14ce381d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:31.383583 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4a95a-f74f-4784-804a-85c14ce381d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:31.383971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4a95a-f74f-4784-804a-85c14ce381d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:31.384421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf4a95a-f74f-4784-804a-85c14ce381d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:31.385536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4a95a-f74f-4784-804a-85c14ce381d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:31.387472 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf4a95a-f74f-4784-804a-85c14ce381d7 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:31.402623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 902/1804] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:31.406624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4166f-5f2f-4689-95cb-0f4f776f5ef3 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:31.417872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4166f-5f2f-4689-95cb-0f4f776f5ef3 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:31.423807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34013f9c-88c6-48b6-bfaa-c148d8e3a270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:31.424560 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34013f9c-88c6-48b6-bfaa-c148d8e3a270 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:31.424950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34013f9c-88c6-48b6-bfaa-c148d8e3a270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:31.425236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34013f9c-88c6-48b6-bfaa-c148d8e3a270 admin admin] PATH_INFO: `/v3/projects/f65f491d64304d36937af703e90eb303/users/b510df41cfa54c508f5130dfbc49c76b/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:31.439280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:31.443629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34013f9c-88c6-48b6-bfaa-c148d8e3a270 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f65f491d64304d36937af703e90eb303, user_id=b510df41cfa54c508f5130dfbc49c76b, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:31.448730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34013f9c-88c6-48b6-bfaa-c148d8e3a270 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:31.472913 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 903/1805] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:31 2022] PUT /identity/v3/projects/f65f491d64304d36937af703e90eb303/users/b510df41cfa54c508f5130dfbc49c76b/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:31.474347 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4a95a-f74f-4784-804a-85c14ce381d7 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:31.485866 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf4a95a-f74f-4784-804a-85c14ce381d7 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:31.551058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98437a-49d2-4c60-9194-1102658193e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:31.551658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98437a-49d2-4c60-9194-1102658193e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:31.552070 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98437a-49d2-4c60-9194-1102658193e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:31.552341 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98437a-49d2-4c60-9194-1102658193e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:31.553357 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf98437a-49d2-4c60-9194-1102658193e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:31.555513 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf98437a-49d2-4c60-9194-1102658193e8 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:31.568665 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 903/1806] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:31.573484 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34013f9c-88c6-48b6-bfaa-c148d8e3a270 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:31.584038 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34013f9c-88c6-48b6-bfaa-c148d8e3a270 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:31.589105 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b7cb83-b10e-4671-93ec-4cf3963f8afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:31.589765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7cb83-b10e-4671-93ec-4cf3963f8afa admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:31.590044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7cb83-b10e-4671-93ec-4cf3963f8afa admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:31.590316 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b7cb83-b10e-4671-93ec-4cf3963f8afa admin admin] PATH_INFO: `/v3/projects/f65f491d64304d36937af703e90eb303/users/b510df41cfa54c508f5130dfbc49c76b/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:31.592600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7cb83-b10e-4671-93ec-4cf3963f8afa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f65f491d64304d36937af703e90eb303, user_id=b510df41cfa54c508f5130dfbc49c76b, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:31.596038 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b7cb83-b10e-4671-93ec-4cf3963f8afa admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:31.608146 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 904/1807] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:31 2022] PUT /identity/v3/projects/f65f491d64304d36937af703e90eb303/users/b510df41cfa54c508f5130dfbc49c76b/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:31.626481 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169c051-b996-4e73-b463-fcff7c83f20c None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:31.627049 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169c051-b996-4e73-b463-fcff7c83f20c None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:31.628111 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169c051-b996-4e73-b463-fcff7c83f20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:31.683464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-2169c051-b996-4e73-b463-fcff7c83f20c None None] MFA Rules not processed for user `b510df41cfa54c508f5130dfbc49c76b`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:31.684831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2169c051-b996-4e73-b463-fcff7c83f20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:31.776091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:31.795509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 904/1808] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:31 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:31.800957 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bbf67-4d0f-4574-a276-9c17f0063010 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:31.802140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bbf67-4d0f-4574-a276-9c17f0063010 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:31.802140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bbf67-4d0f-4574-a276-9c17f0063010 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:31.803977 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 905/1809] 10.222.0.45 () {54 vars in 921 bytes} [Thu Aug 4 02:17:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:17:31.809468 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169c051-b996-4e73-b463-fcff7c83f20c None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:31.812130 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2169c051-b996-4e73-b463-fcff7c83f20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:31.862734 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83f275-0fa2-456b-9796-31617f697a36 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:31.863284 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83f275-0fa2-456b-9796-31617f697a36 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:31.864094 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83f275-0fa2-456b-9796-31617f697a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:31.896961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-4d83f275-0fa2-456b-9796-31617f697a36 None None] MFA Rules not processed for user `c571406d2a3f40dab035a918399d6f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:31.900329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4d83f275-0fa2-456b-9796-31617f697a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:31.900810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169c051-b996-4e73-b463-fcff7c83f20c None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:31.906000 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f70a46-07f6-4204-bfbe-8ea77271d2ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:31.906720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f70a46-07f6-4204-bfbe-8ea77271d2ca service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:31.907211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f70a46-07f6-4204-bfbe-8ea77271d2ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:31.907652 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f70a46-07f6-4204-bfbe-8ea77271d2ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:31.981168 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f70a46-07f6-4204-bfbe-8ea77271d2ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:31.983429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f70a46-07f6-4204-bfbe-8ea77271d2ca service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:31.994559 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 906/1810] 10.222.0.45 () {58 vars in 1018 bytes} [Thu Aug 4 02:17:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:31.998827 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 905/1811] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:32.310081 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83f275-0fa2-456b-9796-31617f697a36 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:32.311594 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4d83f275-0fa2-456b-9796-31617f697a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:32.341067 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83f275-0fa2-456b-9796-31617f697a36 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:32.349296 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c963e8-1f6f-4fe1-aa29-5926e9eec2b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:32.350015 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c963e8-1f6f-4fe1-aa29-5926e9eec2b0 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:32.350839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c963e8-1f6f-4fe1-aa29-5926e9eec2b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:32.350839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c963e8-1f6f-4fe1-aa29-5926e9eec2b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:32.369519 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c963e8-1f6f-4fe1-aa29-5926e9eec2b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:32.373817 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c963e8-1f6f-4fe1-aa29-5926e9eec2b0 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:32.399690 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 907/1812] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:39.751023 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f70a46-07f6-4204-bfbe-8ea77271d2ca service cinder] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:39.759581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f70a46-07f6-4204-bfbe-8ea77271d2ca service cinder] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:39.763480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31131f-9882-43a0-865d-d5ed8bb5b1cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:39.764094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31131f-9882-43a0-865d-d5ed8bb5b1cd service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:39.764374 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31131f-9882-43a0-865d-d5ed8bb5b1cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:39.764670 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31131f-9882-43a0-865d-d5ed8bb5b1cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:39.774467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31131f-9882-43a0-865d-d5ed8bb5b1cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:39.775894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c31131f-9882-43a0-865d-d5ed8bb5b1cd service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:39.785658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 906/1813] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:40.578312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecda9ad-7641-4fae-9d3a-c3cb2c4629f2 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:40.578854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecda9ad-7641-4fae-9d3a-c3cb2c4629f2 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:40.578854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecda9ad-7641-4fae-9d3a-c3cb2c4629f2 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:40.580279 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 908/1814] 10.222.0.45 () {54 vars in 921 bytes} [Thu Aug 4 02:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:17:40.594853 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8d58b-b722-4072-b656-d26ecde4ebef None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:40.595149 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8d58b-b722-4072-b656-d26ecde4ebef None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:40.595430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8d58b-b722-4072-b656-d26ecde4ebef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:40.631047 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-39f8d58b-b722-4072-b656-d26ecde4ebef None None] MFA Rules not processed for user `c571406d2a3f40dab035a918399d6f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:40.632333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-39f8d58b-b722-4072-b656-d26ecde4ebef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:40.644037 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 907/1815] 10.222.0.45 () {58 vars in 1018 bytes} [Thu Aug 4 02:17:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:41.363963 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecda9ad-7641-4fae-9d3a-c3cb2c4629f2 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:41.411040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecda9ad-7641-4fae-9d3a-c3cb2c4629f2 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:41.418953 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2d097-7988-4a4a-8715-1e3d6b3ca2f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:41.419592 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2d097-7988-4a4a-8715-1e3d6b3ca2f4 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:41.419873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2d097-7988-4a4a-8715-1e3d6b3ca2f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:41.420149 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2d097-7988-4a4a-8715-1e3d6b3ca2f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:41.433701 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2d097-7988-4a4a-8715-1e3d6b3ca2f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:41.435817 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2d097-7988-4a4a-8715-1e3d6b3ca2f4 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:41.479861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 909/1816] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:43.895151 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8d58b-b722-4072-b656-d26ecde4ebef None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:43.896710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-39f8d58b-b722-4072-b656-d26ecde4ebef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:44.001540 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f8d58b-b722-4072-b656-d26ecde4ebef None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:44.008502 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb8115-fb46-4993-918e-a2e338fdfa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:44.009437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb8115-fb46-4993-918e-a2e338fdfa62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:44.009932 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb8115-fb46-4993-918e-a2e338fdfa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:44.010785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb8115-fb46-4993-918e-a2e338fdfa62 admin admin] PATH_INFO: `/v3/users/97a6162bf9714d6eb101896535fc71f3` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:44.012246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb8115-fb46-4993-918e-a2e338fdfa62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97a6162bf9714d6eb101896535fc71f3)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:44.014426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb8115-fb46-4993-918e-a2e338fdfa62 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:44.056613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:44.071901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c1cb8115-fb46-4993-918e-a2e338fdfa62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97a6162bf9714d6eb101896535fc71f3'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:44.072562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c1cb8115-fb46-4993-918e-a2e338fdfa62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97a6162bf9714d6eb101896535fc71f3'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:44.073211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c1cb8115-fb46-4993-918e-a2e338fdfa62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97a6162bf9714d6eb101896535fc71f3'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:44.086595 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c1cb8115-fb46-4993-918e-a2e338fdfa62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97a6162bf9714d6eb101896535fc71f3'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:44.094976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c1cb8115-fb46-4993-918e-a2e338fdfa62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97a6162bf9714d6eb101896535fc71f3'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:44.112147 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 908/1817] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:43 2022] DELETE /identity/v3/users/97a6162bf9714d6eb101896535fc71f3 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:44.118315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e125d-cdad-4e27-b690-8c8673b63844 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:44.118863 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e125d-cdad-4e27-b690-8c8673b63844 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:44.118953 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e125d-cdad-4e27-b690-8c8673b63844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:44.139715 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-f93e125d-cdad-4e27-b690-8c8673b63844 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:44.141090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f93e125d-cdad-4e27-b690-8c8673b63844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:44.232668 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 910/1818] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:44.240766 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb8115-fb46-4993-918e-a2e338fdfa62 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:44.243572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-c1cb8115-fb46-4993-918e-a2e338fdfa62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:44.335422 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb8115-fb46-4993-918e-a2e338fdfa62 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:44.340112 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444c1c8-b97f-44c4-b1a2-b42c4c97b253 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:44.341062 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444c1c8-b97f-44c4-b1a2-b42c4c97b253 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:44.341062 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444c1c8-b97f-44c4-b1a2-b42c4c97b253 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:44.341368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444c1c8-b97f-44c4-b1a2-b42c4c97b253 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:44.351982 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444c1c8-b97f-44c4-b1a2-b42c4c97b253 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:44.354192 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444c1c8-b97f-44c4-b1a2-b42c4c97b253 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:44.365564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 909/1819] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:44.769828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e125d-cdad-4e27-b690-8c8673b63844 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:44.771333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f93e125d-cdad-4e27-b690-8c8673b63844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:44.793291 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e125d-cdad-4e27-b690-8c8673b63844 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:44.811741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078cae67-7c26-403b-921b-70f35a6e5efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:44.812405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078cae67-7c26-403b-921b-70f35a6e5efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:44.812722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078cae67-7c26-403b-921b-70f35a6e5efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:44.813004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078cae67-7c26-403b-921b-70f35a6e5efd admin admin] PATH_INFO: `/v3/projects/e5ba8efdcf7f43fe91a00cfae0fb12bd` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:44.814284 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078cae67-7c26-403b-921b-70f35a6e5efd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5ba8efdcf7f43fe91a00cfae0fb12bd)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:44.818343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078cae67-7c26-403b-921b-70f35a6e5efd admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:44.919946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:44.931925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-078cae67-7c26-403b-921b-70f35a6e5efd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5ba8efdcf7f43fe91a00cfae0fb12bd', 'request_id': 'req-078cae67-7c26-403b-921b-70f35a6e5efd'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:44.947951 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-078cae67-7c26-403b-921b-70f35a6e5efd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5ba8efdcf7f43fe91a00cfae0fb12bd', 'request_id': 'req-078cae67-7c26-403b-921b-70f35a6e5efd'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:44.962935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-078cae67-7c26-403b-921b-70f35a6e5efd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5ba8efdcf7f43fe91a00cfae0fb12bd', 'request_id': 'req-078cae67-7c26-403b-921b-70f35a6e5efd'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:44.977704 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-078cae67-7c26-403b-921b-70f35a6e5efd admin admin] The token cache is being invalidate because project e5ba8efdcf7f43fe91a00cfae0fb12bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:44.978408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-078cae67-7c26-403b-921b-70f35a6e5efd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:44.979222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-078cae67-7c26-403b-921b-70f35a6e5efd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:44.981269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 911/1820] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:44 2022] DELETE /identity/v3/projects/e5ba8efdcf7f43fe91a00cfae0fb12bd => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:44.985452 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444c1c8-b97f-44c4-b1a2-b42c4c97b253 service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:44.999752 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444c1c8-b97f-44c4-b1a2-b42c4c97b253 service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:45.104931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078cae67-7c26-403b-921b-70f35a6e5efd admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:45.107138 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-078cae67-7c26-403b-921b-70f35a6e5efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:45.196700 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038c554-e5f6-49fe-8403-a773fe569763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:45.197704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038c554-e5f6-49fe-8403-a773fe569763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:45.198986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038c554-e5f6-49fe-8403-a773fe569763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:45.200986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038c554-e5f6-49fe-8403-a773fe569763 admin admin] PATH_INFO: `/v3/users/7c745d50ea8a43df90e73380695d3c32` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:45.205328 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038c554-e5f6-49fe-8403-a773fe569763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c745d50ea8a43df90e73380695d3c32)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:45.214493 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038c554-e5f6-49fe-8403-a773fe569763 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:45.219082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078cae67-7c26-403b-921b-70f35a6e5efd admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:45.227869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d86a11-17db-406f-8b11-9b49f5acb6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:45.229414 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d86a11-17db-406f-8b11-9b49f5acb6ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:45.229847 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d86a11-17db-406f-8b11-9b49f5acb6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:45.230264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d86a11-17db-406f-8b11-9b49f5acb6ed admin admin] PATH_INFO: `/v3/users/4d0b9d0be26c46d78aa962c49f8dd718` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:45.231931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d86a11-17db-406f-8b11-9b49f5acb6ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d0b9d0be26c46d78aa962c49f8dd718)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:45.235730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d86a11-17db-406f-8b11-9b49f5acb6ed admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:45.322431 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d5d86a11-17db-406f-8b11-9b49f5acb6ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d0b9d0be26c46d78aa962c49f8dd718'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:45.326365 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e038c554-e5f6-49fe-8403-a773fe569763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c745d50ea8a43df90e73380695d3c32'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:45.327323 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e038c554-e5f6-49fe-8403-a773fe569763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c745d50ea8a43df90e73380695d3c32'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:45.328396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e038c554-e5f6-49fe-8403-a773fe569763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c745d50ea8a43df90e73380695d3c32'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:45.339081 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d5d86a11-17db-406f-8b11-9b49f5acb6ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d0b9d0be26c46d78aa962c49f8dd718'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:45.340421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:45.342452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d5d86a11-17db-406f-8b11-9b49f5acb6ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d0b9d0be26c46d78aa962c49f8dd718'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:45.356476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e038c554-e5f6-49fe-8403-a773fe569763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c745d50ea8a43df90e73380695d3c32'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:45.370461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d5d86a11-17db-406f-8b11-9b49f5acb6ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d0b9d0be26c46d78aa962c49f8dd718'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:45.372131 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e038c554-e5f6-49fe-8403-a773fe569763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c745d50ea8a43df90e73380695d3c32'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:45.372825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d5d86a11-17db-406f-8b11-9b49f5acb6ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d0b9d0be26c46d78aa962c49f8dd718'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:45.400743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 912/1821] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:45 2022] DELETE /identity/v3/users/4d0b9d0be26c46d78aa962c49f8dd718 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:45.401078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 910/1822] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:44 2022] DELETE /identity/v3/users/7c745d50ea8a43df90e73380695d3c32 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:45.420874 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d3443-efc5-4601-a27a-1958344e5838 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:45.422945 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d3443-efc5-4601-a27a-1958344e5838 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:45.423364 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d3443-efc5-4601-a27a-1958344e5838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:45.492894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-609d3443-efc5-4601-a27a-1958344e5838 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:45.496912 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-609d3443-efc5-4601-a27a-1958344e5838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:45.598915 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:45.635252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 913/1823] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:45.689810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038c554-e5f6-49fe-8403-a773fe569763 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:45.691272 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e038c554-e5f6-49fe-8403-a773fe569763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:45.767653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038c554-e5f6-49fe-8403-a773fe569763 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:45.778999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97367d16-f36b-4139-99c1-9d92c3c8176b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:45.779767 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97367d16-f36b-4139-99c1-9d92c3c8176b service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:45.780057 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97367d16-f36b-4139-99c1-9d92c3c8176b service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:45.780341 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97367d16-f36b-4139-99c1-9d92c3c8176b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:45.794543 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97367d16-f36b-4139-99c1-9d92c3c8176b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:45.797446 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97367d16-f36b-4139-99c1-9d92c3c8176b service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:45.811647 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 911/1824] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:46.186556 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d3443-efc5-4601-a27a-1958344e5838 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:46.188045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-609d3443-efc5-4601-a27a-1958344e5838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:46.217486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d3443-efc5-4601-a27a-1958344e5838 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:46.225367 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c108e8d-1f74-4fa8-a5d6-55f465240a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:46.226056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c108e8d-1f74-4fa8-a5d6-55f465240a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:46.226300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c108e8d-1f74-4fa8-a5d6-55f465240a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:46.226752 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c108e8d-1f74-4fa8-a5d6-55f465240a9c admin admin] PATH_INFO: `/v3/projects/6d02265f564e46f4b0d1cd2a42c9d167` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:46.228399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c108e8d-1f74-4fa8-a5d6-55f465240a9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d02265f564e46f4b0d1cd2a42c9d167)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:46.233011 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c108e8d-1f74-4fa8-a5d6-55f465240a9c admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:46.300647 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4c108e8d-1f74-4fa8-a5d6-55f465240a9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d02265f564e46f4b0d1cd2a42c9d167', 'request_id': 'req-4c108e8d-1f74-4fa8-a5d6-55f465240a9c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:46.321678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4c108e8d-1f74-4fa8-a5d6-55f465240a9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d02265f564e46f4b0d1cd2a42c9d167', 'request_id': 'req-4c108e8d-1f74-4fa8-a5d6-55f465240a9c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:46.329978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4c108e8d-1f74-4fa8-a5d6-55f465240a9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d02265f564e46f4b0d1cd2a42c9d167', 'request_id': 'req-4c108e8d-1f74-4fa8-a5d6-55f465240a9c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:46.347579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4c108e8d-1f74-4fa8-a5d6-55f465240a9c admin admin] The token cache is being invalidate because project 6d02265f564e46f4b0d1cd2a42c9d167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:46.347579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4c108e8d-1f74-4fa8-a5d6-55f465240a9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:46.347579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4c108e8d-1f74-4fa8-a5d6-55f465240a9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:46.355066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 914/1825] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:46 2022] DELETE /identity/v3/projects/6d02265f564e46f4b0d1cd2a42c9d167 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:46.359799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54476d57-3100-49f8-bd64-98d8ccfbd3b0 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:46.360317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54476d57-3100-49f8-bd64-98d8ccfbd3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:46.360918 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54476d57-3100-49f8-bd64-98d8ccfbd3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:46.384764 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-54476d57-3100-49f8-bd64-98d8ccfbd3b0 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:46.386135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-54476d57-3100-49f8-bd64-98d8ccfbd3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:46.462301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 912/1826] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:46.470687 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe195c-0d95-4972-bd07-c0ab483e1ca4 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:46.471311 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe195c-0d95-4972-bd07-c0ab483e1ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:46.471703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfe195c-0d95-4972-bd07-c0ab483e1ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:46.491254 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54476d57-3100-49f8-bd64-98d8ccfbd3b0 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:46.493341 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-54476d57-3100-49f8-bd64-98d8ccfbd3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:46.520126 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-6cfe195c-0d95-4972-bd07-c0ab483e1ca4 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:46.521857 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-6cfe195c-0d95-4972-bd07-c0ab483e1ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:46.523905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54476d57-3100-49f8-bd64-98d8ccfbd3b0 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:46.535218 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 915/1827] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:46.537508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0dd49-3413-4a5a-b726-d93c16983f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:46.540021 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0dd49-3413-4a5a-b726-d93c16983f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:46.540420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0dd49-3413-4a5a-b726-d93c16983f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:46.540947 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0dd49-3413-4a5a-b726-d93c16983f00 admin admin] PATH_INFO: `/v3/projects/a850f1723cd645ec95113b4449a32479` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:46.541173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe195c-0d95-4972-bd07-c0ab483e1ca4 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:46.542366 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0dd49-3413-4a5a-b726-d93c16983f00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a850f1723cd645ec95113b4449a32479)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:46.545665 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a0dd49-3413-4a5a-b726-d93c16983f00 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:46.558245 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfe195c-0d95-4972-bd07-c0ab483e1ca4 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:46.563372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e1fdd-40de-4999-adda-1f0ba01be5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:46.564243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e1fdd-40de-4999-adda-1f0ba01be5ac admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:46.564523 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e1fdd-40de-4999-adda-1f0ba01be5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:46.564833 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e1fdd-40de-4999-adda-1f0ba01be5ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:46.565582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e1fdd-40de-4999-adda-1f0ba01be5ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:17:46.565897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e1fdd-40de-4999-adda-1f0ba01be5ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:46.567530 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e1fdd-40de-4999-adda-1f0ba01be5ac admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:46.582129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 916/1828] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:17:46.587140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e1fdd-40de-4999-adda-1f0ba01be5ac admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:46.597137 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e1fdd-40de-4999-adda-1f0ba01be5ac admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:46.609741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d9a0dd49-3413-4a5a-b726-d93c16983f00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a850f1723cd645ec95113b4449a32479', 'request_id': 'req-d9a0dd49-3413-4a5a-b726-d93c16983f00'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:46.623007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d9a0dd49-3413-4a5a-b726-d93c16983f00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a850f1723cd645ec95113b4449a32479', 'request_id': 'req-d9a0dd49-3413-4a5a-b726-d93c16983f00'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:46.632160 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d9a0dd49-3413-4a5a-b726-d93c16983f00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a850f1723cd645ec95113b4449a32479', 'request_id': 'req-d9a0dd49-3413-4a5a-b726-d93c16983f00'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:46.636159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:46.656936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d9a0dd49-3413-4a5a-b726-d93c16983f00 admin admin] The token cache is being invalidate because project a850f1723cd645ec95113b4449a32479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:46.657483 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d9a0dd49-3413-4a5a-b726-d93c16983f00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:46.658257 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d9a0dd49-3413-4a5a-b726-d93c16983f00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:46.662356 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 913/1829] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:46 2022] DELETE /identity/v3/projects/a850f1723cd645ec95113b4449a32479 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:46.666150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0dd49-3413-4a5a-b726-d93c16983f00 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:46.668378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d9a0dd49-3413-4a5a-b726-d93c16983f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:46.685381 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc69a5-db4a-4d30-9493-3699f5c3ae5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:46.686054 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc69a5-db4a-4d30-9493-3699f5c3ae5f admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:46.687454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc69a5-db4a-4d30-9493-3699f5c3ae5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:46.687454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc69a5-db4a-4d30-9493-3699f5c3ae5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:46.687454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc69a5-db4a-4d30-9493-3699f5c3ae5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:46.693450 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc69a5-db4a-4d30-9493-3699f5c3ae5f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:46.766401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 917/1830] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:46 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:46.771103 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc69a5-db4a-4d30-9493-3699f5c3ae5f admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:46.774551 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7efc69a5-db4a-4d30-9493-3699f5c3ae5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:46.787763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0dd49-3413-4a5a-b726-d93c16983f00 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:46.849202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:46.867426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35bf61-a207-447a-80e7-14f8607747a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:46.868079 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35bf61-a207-447a-80e7-14f8607747a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:46.868357 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35bf61-a207-447a-80e7-14f8607747a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:46.868674 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d35bf61-a207-447a-80e7-14f8607747a6 admin admin] PATH_INFO: `/v3/users/b510df41cfa54c508f5130dfbc49c76b` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:46.870001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35bf61-a207-447a-80e7-14f8607747a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b510df41cfa54c508f5130dfbc49c76b)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:46.872739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d35bf61-a207-447a-80e7-14f8607747a6 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:46.897686 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc69a5-db4a-4d30-9493-3699f5c3ae5f admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:46.905276 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0a182-d8da-4f53-b587-157cfeff4e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:46.905950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0a182-d8da-4f53-b587-157cfeff4e1c admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:46.906506 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0a182-d8da-4f53-b587-157cfeff4e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:46.907500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0a182-d8da-4f53-b587-157cfeff4e1c admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:46.908265 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0a182-d8da-4f53-b587-157cfeff4e1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:46.911513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0a182-d8da-4f53-b587-157cfeff4e1c admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:46.936668 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 918/1831] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:46 2022] POST /identity/v3/users => generated 294 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:17:46.937558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4d35bf61-a207-447a-80e7-14f8607747a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b510df41cfa54c508f5130dfbc49c76b'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:46.938329 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4d35bf61-a207-447a-80e7-14f8607747a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b510df41cfa54c508f5130dfbc49c76b'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:46.938943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4d35bf61-a207-447a-80e7-14f8607747a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b510df41cfa54c508f5130dfbc49c76b'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:46.939412 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0a182-d8da-4f53-b587-157cfeff4e1c admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:46.941014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9ba0a182-d8da-4f53-b587-157cfeff4e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:46.950955 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4d35bf61-a207-447a-80e7-14f8607747a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b510df41cfa54c508f5130dfbc49c76b'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:46.952908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0a182-d8da-4f53-b587-157cfeff4e1c admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:46.958020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d95c32-f706-4366-acf6-f3409d2397a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:46.958495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4d35bf61-a207-447a-80e7-14f8607747a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b510df41cfa54c508f5130dfbc49c76b'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:46.958748 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d95c32-f706-4366-acf6-f3409d2397a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:46.959029 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d95c32-f706-4366-acf6-f3409d2397a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:46.959303 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d95c32-f706-4366-acf6-f3409d2397a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:46.960025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d95c32-f706-4366-acf6-f3409d2397a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:46.962220 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d95c32-f706-4366-acf6-f3409d2397a4 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:46.976356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 919/1832] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:17:46.977042 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 914/1833] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:46 2022] DELETE /identity/v3/users/b510df41cfa54c508f5130dfbc49c76b => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:46.978586 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d95c32-f706-4366-acf6-f3409d2397a4 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:46.988333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d95c32-f706-4366-acf6-f3409d2397a4 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:47.055922 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2de4a-a7bc-472f-85e9-67cafc1487bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:47.057642 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2de4a-a7bc-472f-85e9-67cafc1487bc admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:47.058197 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2de4a-a7bc-472f-85e9-67cafc1487bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:47.058550 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2de4a-a7bc-472f-85e9-67cafc1487bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:47.059418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2de4a-a7bc-472f-85e9-67cafc1487bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:47.062369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2de4a-a7bc-472f-85e9-67cafc1487bc admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:47.074585 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 920/1834] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:47.079056 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35bf61-a207-447a-80e7-14f8607747a6 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:47.092375 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d35bf61-a207-447a-80e7-14f8607747a6 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:47.097449 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a023394-1c66-4149-814f-e846e026ed28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:47.098207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a023394-1c66-4149-814f-e846e026ed28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:47.098489 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a023394-1c66-4149-814f-e846e026ed28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:47.098766 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a023394-1c66-4149-814f-e846e026ed28 admin admin] PATH_INFO: `/v3/projects/a8f2ac8c139b4849a40f4d4504e37272/users/5d7617cb37114f33a12add7a908ac930/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:47.118182 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a023394-1c66-4149-814f-e846e026ed28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8f2ac8c139b4849a40f4d4504e37272, user_id=5d7617cb37114f33a12add7a908ac930, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:47.121763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a023394-1c66-4149-814f-e846e026ed28 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:47.134124 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:47.155942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 915/1835] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:47 2022] PUT /identity/v3/projects/a8f2ac8c139b4849a40f4d4504e37272/users/5d7617cb37114f33a12add7a908ac930/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:47.158338 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb642bb-377c-4cc0-a3c9-8e489b179ecc None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:47.158757 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb642bb-377c-4cc0-a3c9-8e489b179ecc None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:47.159279 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb642bb-377c-4cc0-a3c9-8e489b179ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:47.193422 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-afb642bb-377c-4cc0-a3c9-8e489b179ecc None None] MFA Rules not processed for user `5d7617cb37114f33a12add7a908ac930`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:47.196589 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-afb642bb-377c-4cc0-a3c9-8e489b179ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:47.293895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a023394-1c66-4149-814f-e846e026ed28 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:47.296768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 921/1836] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:47.421764 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a023394-1c66-4149-814f-e846e026ed28 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:47.430879 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6cd22b-d37e-4425-84f6-8b048625e398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:47.433291 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6cd22b-d37e-4425-84f6-8b048625e398 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:47.433586 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6cd22b-d37e-4425-84f6-8b048625e398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:47.433904 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6cd22b-d37e-4425-84f6-8b048625e398 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:47.434821 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6cd22b-d37e-4425-84f6-8b048625e398 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:47.446451 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb642bb-377c-4cc0-a3c9-8e489b179ecc None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:47.448518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-afb642bb-377c-4cc0-a3c9-8e489b179ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:47.469722 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6cd22b-d37e-4425-84f6-8b048625e398 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:47.486753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb642bb-377c-4cc0-a3c9-8e489b179ecc None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:47.491485 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fe7bc6-495d-4fcd-b40c-51c6bd261744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:47.492224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe7bc6-495d-4fcd-b40c-51c6bd261744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:47.492503 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe7bc6-495d-4fcd-b40c-51c6bd261744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:47.492805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fe7bc6-495d-4fcd-b40c-51c6bd261744 admin admin] PATH_INFO: `/v3/projects/f65f491d64304d36937af703e90eb303` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:47.493994 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe7bc6-495d-4fcd-b40c-51c6bd261744 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f65f491d64304d36937af703e90eb303)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:47.497007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fe7bc6-495d-4fcd-b40c-51c6bd261744 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:47.539061 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 916/1837] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:47 2022] POST /identity/v3/projects => generated 387 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:47.544183 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6cd22b-d37e-4425-84f6-8b048625e398 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:47.598267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:47.619839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-26fe7bc6-495d-4fcd-b40c-51c6bd261744 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f65f491d64304d36937af703e90eb303', 'request_id': 'req-26fe7bc6-495d-4fcd-b40c-51c6bd261744'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:47.636450 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-26fe7bc6-495d-4fcd-b40c-51c6bd261744 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f65f491d64304d36937af703e90eb303', 'request_id': 'req-26fe7bc6-495d-4fcd-b40c-51c6bd261744'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:47.648432 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-26fe7bc6-495d-4fcd-b40c-51c6bd261744 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f65f491d64304d36937af703e90eb303', 'request_id': 'req-26fe7bc6-495d-4fcd-b40c-51c6bd261744'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:47.669456 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-26fe7bc6-495d-4fcd-b40c-51c6bd261744 admin admin] The token cache is being invalidate because project f65f491d64304d36937af703e90eb303 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:47.670030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-26fe7bc6-495d-4fcd-b40c-51c6bd261744 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:47.671698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-26fe7bc6-495d-4fcd-b40c-51c6bd261744 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:47.674361 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 922/1838] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:47 2022] DELETE /identity/v3/projects/f65f491d64304d36937af703e90eb303 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:47.680917 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dac9e0-1bca-4210-bb2b-44e9479eb2ad None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:47.681865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dac9e0-1bca-4210-bb2b-44e9479eb2ad None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:47.682296 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dac9e0-1bca-4210-bb2b-44e9479eb2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:47.703511 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6cd22b-d37e-4425-84f6-8b048625e398 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:47.711101 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c4dac9e0-1bca-4210-bb2b-44e9479eb2ad None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:47.713606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c4dac9e0-1bca-4210-bb2b-44e9479eb2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:47.814460 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41feec-8c38-43dd-b484-dfe33fb90eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:47.815200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41feec-8c38-43dd-b484-dfe33fb90eb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:47.816782 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41feec-8c38-43dd-b484-dfe33fb90eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:47.817822 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41feec-8c38-43dd-b484-dfe33fb90eb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:47.819011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41feec-8c38-43dd-b484-dfe33fb90eb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:47.822322 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41feec-8c38-43dd-b484-dfe33fb90eb3 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:47.839196 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 923/1839] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:47.847198 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50082bf6-63a2-4a7b-9bbd-7b4cebf623c8 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:47.847577 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50082bf6-63a2-4a7b-9bbd-7b4cebf623c8 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:47.847931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50082bf6-63a2-4a7b-9bbd-7b4cebf623c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:47.856773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 917/1840] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:47 2022] POST /identity/v3/users => generated 300 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:47.859874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41feec-8c38-43dd-b484-dfe33fb90eb3 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:47.868753 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-5a41feec-8c38-43dd-b484-dfe33fb90eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:47.882807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41feec-8c38-43dd-b484-dfe33fb90eb3 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:47.887188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-50082bf6-63a2-4a7b-9bbd-7b4cebf623c8 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:47.888402 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-50082bf6-63a2-4a7b-9bbd-7b4cebf623c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:47.889859 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9c6a8-05a3-45c1-8766-bb4f1739ebf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:47.891732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9c6a8-05a3-45c1-8766-bb4f1739ebf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:47.892121 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9c6a8-05a3-45c1-8766-bb4f1739ebf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:47.892567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b9c6a8-05a3-45c1-8766-bb4f1739ebf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:47.894326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9c6a8-05a3-45c1-8766-bb4f1739ebf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:47.897380 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b9c6a8-05a3-45c1-8766-bb4f1739ebf8 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:47.901047 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 924/1841] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:47.902682 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50082bf6-63a2-4a7b-9bbd-7b4cebf623c8 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:47.914764 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50082bf6-63a2-4a7b-9bbd-7b4cebf623c8 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:47.914965 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 918/1842] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:47.917834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9c6a8-05a3-45c1-8766-bb4f1739ebf8 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:47.921384 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e606ea8-f100-4e3e-80ab-3e3e2b43cbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:47.921991 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e606ea8-f100-4e3e-80ab-3e3e2b43cbc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:47.922269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e606ea8-f100-4e3e-80ab-3e3e2b43cbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:47.922555 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e606ea8-f100-4e3e-80ab-3e3e2b43cbc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:47.929764 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b9c6a8-05a3-45c1-8766-bb4f1739ebf8 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:47.934768 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f409274-99c2-44fe-a0cd-f8aa0dcb9058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:47.935309 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e606ea8-f100-4e3e-80ab-3e3e2b43cbc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:17:47.935633 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e606ea8-f100-4e3e-80ab-3e3e2b43cbc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:47.935836 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f409274-99c2-44fe-a0cd-f8aa0dcb9058 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:47.936191 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f409274-99c2-44fe-a0cd-f8aa0dcb9058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:47.936541 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f409274-99c2-44fe-a0cd-f8aa0dcb9058 admin admin] PATH_INFO: `/v3/projects/094143dbf2b048b0bc0b32af456608c4/users/56b423366d844479909fd50981a19334/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:47.937295 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e606ea8-f100-4e3e-80ab-3e3e2b43cbc6 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:47.940464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:47.957724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f409274-99c2-44fe-a0cd-f8aa0dcb9058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=094143dbf2b048b0bc0b32af456608c4, user_id=56b423366d844479909fd50981a19334, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:47.961520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f409274-99c2-44fe-a0cd-f8aa0dcb9058 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:47.974637 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 925/1843] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:17:47.979274 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e606ea8-f100-4e3e-80ab-3e3e2b43cbc6 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:47.987526 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 919/1844] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:47 2022] PUT /identity/v3/projects/094143dbf2b048b0bc0b32af456608c4/users/56b423366d844479909fd50981a19334/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:47.991946 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f409274-99c2-44fe-a0cd-f8aa0dcb9058 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:47.998027 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e606ea8-f100-4e3e-80ab-3e3e2b43cbc6 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:48.077846 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f409274-99c2-44fe-a0cd-f8aa0dcb9058 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:48.078048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4f171-a04a-4ef3-9b31-67a0725cdd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:48.078607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4f171-a04a-4ef3-9b31-67a0725cdd83 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:48.078890 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4f171-a04a-4ef3-9b31-67a0725cdd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:48.079167 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd4f171-a04a-4ef3-9b31-67a0725cdd83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:48.080013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4f171-a04a-4ef3-9b31-67a0725cdd83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:48.082614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd4f171-a04a-4ef3-9b31-67a0725cdd83 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:48.085080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ba69a-66b9-4531-b245-cfffaab757e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:48.086246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ba69a-66b9-4531-b245-cfffaab757e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:48.086558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ba69a-66b9-4531-b245-cfffaab757e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:48.086830 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ba69a-66b9-4531-b245-cfffaab757e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:48.087621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ba69a-66b9-4531-b245-cfffaab757e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:48.089904 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ba69a-66b9-4531-b245-cfffaab757e8 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:48.101318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 920/1845] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:17:48.105850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ba69a-66b9-4531-b245-cfffaab757e8 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:48.116085 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ba69a-66b9-4531-b245-cfffaab757e8 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:48.120999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 926/1846] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:47 2022] POST /identity/v3/projects => generated 377 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:48.122057 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4f171-a04a-4ef3-9b31-67a0725cdd83 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:48.157969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:48.195959 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd4f171-a04a-4ef3-9b31-67a0725cdd83 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:48.197291 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa0495-20b9-4cd4-b146-04f15bb3f99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:48.197963 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa0495-20b9-4cd4-b146-04f15bb3f99c admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:48.198244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa0495-20b9-4cd4-b146-04f15bb3f99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:48.198547 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa0495-20b9-4cd4-b146-04f15bb3f99c admin admin] PATH_INFO: `/v3/projects/094143dbf2b048b0bc0b32af456608c4/users/56b423366d844479909fd50981a19334/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:48.200760 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa0495-20b9-4cd4-b146-04f15bb3f99c admin admin] RBAC: Authorizing `identity:create_grant(project_id=094143dbf2b048b0bc0b32af456608c4, user_id=56b423366d844479909fd50981a19334, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:48.200932 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d31ea7-8c91-4f83-9d7d-effb0a7f4bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:48.201506 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d31ea7-8c91-4f83-9d7d-effb0a7f4bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:48.201894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d31ea7-8c91-4f83-9d7d-effb0a7f4bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:48.202170 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d31ea7-8c91-4f83-9d7d-effb0a7f4bd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:48.202814 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d31ea7-8c91-4f83-9d7d-effb0a7f4bd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:48.203520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aa0495-20b9-4cd4-b146-04f15bb3f99c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:48.204785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d31ea7-8c91-4f83-9d7d-effb0a7f4bd5 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:48.216856 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 921/1847] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:48 2022] PUT /identity/v3/projects/094143dbf2b048b0bc0b32af456608c4/users/56b423366d844479909fd50981a19334/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:48.228567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7993c-328a-4561-aa72-a6ef841da590 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:48.228963 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7993c-328a-4561-aa72-a6ef841da590 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:48.229259 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7993c-328a-4561-aa72-a6ef841da590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:48.230414 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 927/1848] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:48 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:17:48.234602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d31ea7-8c91-4f83-9d7d-effb0a7f4bd5 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:48.246476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d31ea7-8c91-4f83-9d7d-effb0a7f4bd5 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:48.264664 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-c5b7993c-328a-4561-aa72-a6ef841da590 None None] MFA Rules not processed for user `56b423366d844479909fd50981a19334`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:48.265893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-c5b7993c-328a-4561-aa72-a6ef841da590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:48.311019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c4e5e-00e3-4baf-8e70-94a119d11c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:48.311651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c4e5e-00e3-4baf-8e70-94a119d11c8e admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:48.312061 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c4e5e-00e3-4baf-8e70-94a119d11c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:48.312399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c4e5e-00e3-4baf-8e70-94a119d11c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:48.313218 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c4e5e-00e3-4baf-8e70-94a119d11c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:48.314991 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c4e5e-00e3-4baf-8e70-94a119d11c8e admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:48.325447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 928/1849] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:48.329812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c4e5e-00e3-4baf-8e70-94a119d11c8e admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:48.347241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c4e5e-00e3-4baf-8e70-94a119d11c8e admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:48.355767 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83d36f-ed2e-4f2b-91ae-a08bcc83d5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:48.356759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83d36f-ed2e-4f2b-91ae-a08bcc83d5b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:48.357211 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83d36f-ed2e-4f2b-91ae-a08bcc83d5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:48.357663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83d36f-ed2e-4f2b-91ae-a08bcc83d5b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:48.358842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83d36f-ed2e-4f2b-91ae-a08bcc83d5b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:48.361666 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83d36f-ed2e-4f2b-91ae-a08bcc83d5b6 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:48.364085 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:48.371778 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 922/1850] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:48.381330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 929/1851] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:17:48.387230 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7993c-328a-4561-aa72-a6ef841da590 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:48.388136 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83d36f-ed2e-4f2b-91ae-a08bcc83d5b6 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:48.390591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-0d83d36f-ed2e-4f2b-91ae-a08bcc83d5b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:48.399641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7993c-328a-4561-aa72-a6ef841da590 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:48.404519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8114e-3eb3-40ca-8a82-01690d448733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:48.405247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8114e-3eb3-40ca-8a82-01690d448733 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:48.406069 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8114e-3eb3-40ca-8a82-01690d448733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:48.406069 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8114e-3eb3-40ca-8a82-01690d448733 admin admin] PATH_INFO: `/v3/projects/37ae7a75818544b2809031243e85749a/users/9eb9b935f4324c26b7f7ad7ff256630c/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:48.424646 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8114e-3eb3-40ca-8a82-01690d448733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ae7a75818544b2809031243e85749a, user_id=9eb9b935f4324c26b7f7ad7ff256630c, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:48.427539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8114e-3eb3-40ca-8a82-01690d448733 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:48.457992 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 923/1852] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:48 2022] PUT /identity/v3/projects/37ae7a75818544b2809031243e85749a/users/9eb9b935f4324c26b7f7ad7ff256630c/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:48.463711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9311a6-0d97-48e7-bfec-c63e8e1ca0aa None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:48.464001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9311a6-0d97-48e7-bfec-c63e8e1ca0aa None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:48.464272 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9311a6-0d97-48e7-bfec-c63e8e1ca0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:48.491127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83d36f-ed2e-4f2b-91ae-a08bcc83d5b6 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:48.495069 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-0b9311a6-0d97-48e7-bfec-c63e8e1ca0aa None None] MFA Rules not processed for user `9eb9b935f4324c26b7f7ad7ff256630c`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:48.500855 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-0b9311a6-0d97-48e7-bfec-c63e8e1ca0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:48.559150 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71858a8c-a315-41a8-b826-b3b55db6f094 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:48.559494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71858a8c-a315-41a8-b826-b3b55db6f094 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:48.559781 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71858a8c-a315-41a8-b826-b3b55db6f094 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:48.561066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71858a8c-a315-41a8-b826-b3b55db6f094 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:48.586198 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:48.609340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 924/1853] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:48.614262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9311a6-0d97-48e7-bfec-c63e8e1ca0aa None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:48.665277 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71858a8c-a315-41a8-b826-b3b55db6f094 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:48.668374 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71858a8c-a315-41a8-b826-b3b55db6f094 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:48.686769 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 930/1854] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:48.758606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9311a6-0d97-48e7-bfec-c63e8e1ca0aa None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:48.765599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f67f0-e1bd-4b21-86e5-a06631958d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:48.766348 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f67f0-e1bd-4b21-86e5-a06631958d0c admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:48.766942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f67f0-e1bd-4b21-86e5-a06631958d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:48.767325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f67f0-e1bd-4b21-86e5-a06631958d0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:48.768501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f67f0-e1bd-4b21-86e5-a06631958d0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:48.771142 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f67f0-e1bd-4b21-86e5-a06631958d0c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:48.821154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71858a8c-a315-41a8-b826-b3b55db6f094 service cinder] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:48.825917 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 925/1855] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:48 2022] POST /identity/v3/projects => generated 375 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:17:48.830985 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f67f0-e1bd-4b21-86e5-a06631958d0c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:48.834765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71858a8c-a315-41a8-b826-b3b55db6f094 service cinder] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:48.915499 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9c94d-b470-41ef-a53c-72169424c97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:48.916241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9c94d-b470-41ef-a53c-72169424c97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:48.916592 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9c94d-b470-41ef-a53c-72169424c97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:48.916966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9c94d-b470-41ef-a53c-72169424c97b admin admin] PATH_INFO: `/v3/users/5d7617cb37114f33a12add7a908ac930` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:48.918371 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9c94d-b470-41ef-a53c-72169424c97b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d7617cb37114f33a12add7a908ac930)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:48.920709 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9c94d-b470-41ef-a53c-72169424c97b admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:48.955845 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:48.985620 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f67f0-e1bd-4b21-86e5-a06631958d0c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:48.993538 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ec34e-73f0-48ec-b1cb-9f0a1fc91fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:48.994707 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ec34e-73f0-48ec-b1cb-9f0a1fc91fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:48.995738 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ec34e-73f0-48ec-b1cb-9f0a1fc91fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:48.996210 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ec34e-73f0-48ec-b1cb-9f0a1fc91fa3 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:48.997132 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ec34e-73f0-48ec-b1cb-9f0a1fc91fa3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:48.999929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ec34e-73f0-48ec-b1cb-9f0a1fc91fa3 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:49.016712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8db9c94d-b470-41ef-a53c-72169424c97b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d7617cb37114f33a12add7a908ac930'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:49.029753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8db9c94d-b470-41ef-a53c-72169424c97b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d7617cb37114f33a12add7a908ac930'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:49.030917 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8db9c94d-b470-41ef-a53c-72169424c97b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d7617cb37114f33a12add7a908ac930'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:49.035656 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 926/1856] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:48 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:17:49.036654 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ec34e-73f0-48ec-b1cb-9f0a1fc91fa3 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:49.051245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ec34e-73f0-48ec-b1cb-9f0a1fc91fa3 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:49.078722 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf17536-6bc8-49d4-88b2-517d152bcd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:49.079631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8db9c94d-b470-41ef-a53c-72169424c97b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d7617cb37114f33a12add7a908ac930'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:49.079902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf17536-6bc8-49d4-88b2-517d152bcd5f admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:49.080357 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8db9c94d-b470-41ef-a53c-72169424c97b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d7617cb37114f33a12add7a908ac930'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:49.082102 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf17536-6bc8-49d4-88b2-517d152bcd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:49.082776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf17536-6bc8-49d4-88b2-517d152bcd5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:49.087689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf17536-6bc8-49d4-88b2-517d152bcd5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:49.094465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf17536-6bc8-49d4-88b2-517d152bcd5f admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:49.105067 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 931/1857] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:48 2022] DELETE /identity/v3/users/5d7617cb37114f33a12add7a908ac930 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:49.109762 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e683c-d10e-41bf-972d-3f4d4435ccaf None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:49.110399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e683c-d10e-41bf-972d-3f4d4435ccaf None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:49.110846 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e683c-d10e-41bf-972d-3f4d4435ccaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:49.121919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 927/1858] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:49.128821 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf17536-6bc8-49d4-88b2-517d152bcd5f admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:49.135858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-caf17536-6bc8-49d4-88b2-517d152bcd5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:49.156396 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c87e683c-d10e-41bf-972d-3f4d4435ccaf None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:49.159923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c87e683c-d10e-41bf-972d-3f4d4435ccaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:49.167760 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf17536-6bc8-49d4-88b2-517d152bcd5f admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:49.213452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:49.264228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3e562-c25b-4c25-8c19-9a89482e090e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:49.265077 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3e562-c25b-4c25-8c19-9a89482e090e admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:49.265424 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3e562-c25b-4c25-8c19-9a89482e090e admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:49.265780 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3e562-c25b-4c25-8c19-9a89482e090e admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:49.266569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3e562-c25b-4c25-8c19-9a89482e090e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:49.268742 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3e562-c25b-4c25-8c19-9a89482e090e admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:49.272354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 932/1859] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:49.284051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 928/1860] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:17:49.285740 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e683c-d10e-41bf-972d-3f4d4435ccaf None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:49.296844 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e683c-d10e-41bf-972d-3f4d4435ccaf None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:49.301977 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a882540-a238-411e-b2a7-a6343c86467d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:49.302759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a882540-a238-411e-b2a7-a6343c86467d admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:49.303101 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a882540-a238-411e-b2a7-a6343c86467d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:49.303469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a882540-a238-411e-b2a7-a6343c86467d admin admin] PATH_INFO: `/v3/projects/3ee3e0b03cfc44dc8b321951feef3a61/users/d9885632a3ac46779e475ddbffe3ef17/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:49.311299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3e562-c25b-4c25-8c19-9a89482e090e admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:49.313186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-02a3e562-c25b-4c25-8c19-9a89482e090e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:49.325463 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a882540-a238-411e-b2a7-a6343c86467d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee3e0b03cfc44dc8b321951feef3a61, user_id=d9885632a3ac46779e475ddbffe3ef17, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:49.328698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a882540-a238-411e-b2a7-a6343c86467d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:49.353242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 933/1861] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:49 2022] PUT /identity/v3/projects/3ee3e0b03cfc44dc8b321951feef3a61/users/d9885632a3ac46779e475ddbffe3ef17/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:49.364457 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415de4b3-64b2-4ed8-9ea7-1fc9b872cbaf None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:49.365399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415de4b3-64b2-4ed8-9ea7-1fc9b872cbaf None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:49.365399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415de4b3-64b2-4ed8-9ea7-1fc9b872cbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:49.419795 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-415de4b3-64b2-4ed8-9ea7-1fc9b872cbaf None None] MFA Rules not processed for user `d9885632a3ac46779e475ddbffe3ef17`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:49.421321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-415de4b3-64b2-4ed8-9ea7-1fc9b872cbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:49.470248 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3e562-c25b-4c25-8c19-9a89482e090e admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:49.521456 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 934/1862] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:49.529664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415de4b3-64b2-4ed8-9ea7-1fc9b872cbaf None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:49.535202 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249c133-b186-4a60-861c-d51e5f4a322c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:49.536386 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249c133-b186-4a60-861c-d51e5f4a322c service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:49.536839 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249c133-b186-4a60-861c-d51e5f4a322c service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:49.537219 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0249c133-b186-4a60-861c-d51e5f4a322c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:49.579157 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:49.621102 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249c133-b186-4a60-861c-d51e5f4a322c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:49.623710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0249c133-b186-4a60-861c-d51e5f4a322c service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:49.637108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 929/1863] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:17:49.672323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415de4b3-64b2-4ed8-9ea7-1fc9b872cbaf None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:49.677938 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69605e22-e974-450c-bc57-81ee4a99d3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:49.678982 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69605e22-e974-450c-bc57-81ee4a99d3ac admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:49.679504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69605e22-e974-450c-bc57-81ee4a99d3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:49.680275 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69605e22-e974-450c-bc57-81ee4a99d3ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:49.681320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69605e22-e974-450c-bc57-81ee4a99d3ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:49.683818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69605e22-e974-450c-bc57-81ee4a99d3ac admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:49.724654 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 935/1864] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:49 2022] POST /identity/v3/projects => generated 375 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:49.736370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249c133-b186-4a60-861c-d51e5f4a322c service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:49.748109 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0249c133-b186-4a60-861c-d51e5f4a322c service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:49.843603 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700da79e-043f-4e99-b899-f1a22cb0ffd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:49.845027 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700da79e-043f-4e99-b899-f1a22cb0ffd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:49.846934 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700da79e-043f-4e99-b899-f1a22cb0ffd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:49.847946 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700da79e-043f-4e99-b899-f1a22cb0ffd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:49.853268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700da79e-043f-4e99-b899-f1a22cb0ffd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:49.857533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700da79e-043f-4e99-b899-f1a22cb0ffd1 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:49.907218 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 930/1865] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:49 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:49.911702 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69605e22-e974-450c-bc57-81ee4a99d3ac admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:49.927581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69605e22-e974-450c-bc57-81ee4a99d3ac admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:49.935561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a424a-4485-45ba-988a-5bd90f9c142d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:49.936267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a424a-4485-45ba-988a-5bd90f9c142d admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:49.937790 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a424a-4485-45ba-988a-5bd90f9c142d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:49.938259 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a424a-4485-45ba-988a-5bd90f9c142d admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:49.939034 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a424a-4485-45ba-988a-5bd90f9c142d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:49.941614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a424a-4485-45ba-988a-5bd90f9c142d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:49.977360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 936/1866] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:17:49.978672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700da79e-043f-4e99-b899-f1a22cb0ffd1 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:49.990079 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700da79e-043f-4e99-b899-f1a22cb0ffd1 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:49.997933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888f337-25a4-434f-ae9d-ece7eb129a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:49.998710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888f337-25a4-434f-ae9d-ece7eb129a79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:49.999195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888f337-25a4-434f-ae9d-ece7eb129a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:49.999668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1888f337-25a4-434f-ae9d-ece7eb129a79 admin admin] PATH_INFO: `/v3/projects/c2644fb449d6470291261892b7a46a6c/users/57738349f9f34c12b35d397c55ec9130/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:50.028525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888f337-25a4-434f-ae9d-ece7eb129a79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2644fb449d6470291261892b7a46a6c, user_id=57738349f9f34c12b35d397c55ec9130, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:50.034655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1888f337-25a4-434f-ae9d-ece7eb129a79 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:50.063046 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 931/1867] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:49 2022] PUT /identity/v3/projects/c2644fb449d6470291261892b7a46a6c/users/57738349f9f34c12b35d397c55ec9130/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:50.064311 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a424a-4485-45ba-988a-5bd90f9c142d admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:50.075486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a424a-4485-45ba-988a-5bd90f9c142d admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:50.083875 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:50.174754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bfd63-5056-45d5-a728-eb22749350ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:50.174754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bfd63-5056-45d5-a728-eb22749350ce admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:50.174754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bfd63-5056-45d5-a728-eb22749350ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:50.174754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05bfd63-5056-45d5-a728-eb22749350ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:50.176006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888f337-25a4-434f-ae9d-ece7eb129a79 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:50.178001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-1888f337-25a4-434f-ae9d-ece7eb129a79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:50.183206 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05bfd63-5056-45d5-a728-eb22749350ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:50.189128 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05bfd63-5056-45d5-a728-eb22749350ce admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:50.209576 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1888f337-25a4-434f-ae9d-ece7eb129a79 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:50.210534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 937/1868] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:50.212168 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bfd63-5056-45d5-a728-eb22749350ce admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:50.216526 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:50.217426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:50.217837 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:50.218396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e admin admin] PATH_INFO: `/v3/projects/a8f2ac8c139b4849a40f4d4504e37272` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:50.220167 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8f2ac8c139b4849a40f4d4504e37272)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:50.222262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:50.223236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05bfd63-5056-45d5-a728-eb22749350ce admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:50.229680 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7b1c71-a096-45a8-b5ad-64e4b5e709b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:50.231007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b1c71-a096-45a8-b5ad-64e4b5e709b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:50.231293 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b1c71-a096-45a8-b5ad-64e4b5e709b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:50.231610 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7b1c71-a096-45a8-b5ad-64e4b5e709b8 admin admin] PATH_INFO: `/v3/projects/c2644fb449d6470291261892b7a46a6c/users/57738349f9f34c12b35d397c55ec9130/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:50.235559 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b1c71-a096-45a8-b5ad-64e4b5e709b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2644fb449d6470291261892b7a46a6c, user_id=57738349f9f34c12b35d397c55ec9130, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:50.237567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7b1c71-a096-45a8-b5ad-64e4b5e709b8 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:50.251656 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 938/1869] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:50 2022] PUT /identity/v3/projects/c2644fb449d6470291261892b7a46a6c/users/57738349f9f34c12b35d397c55ec9130/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:50.262850 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459260f-67d7-4f3e-88ec-e369ef6c13bd None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:50.263135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459260f-67d7-4f3e-88ec-e369ef6c13bd None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:50.263412 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459260f-67d7-4f3e-88ec-e369ef6c13bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:50.298022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-7459260f-67d7-4f3e-88ec-e369ef6c13bd None None] MFA Rules not processed for user `57738349f9f34c12b35d397c55ec9130`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:50.298710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8f2ac8c139b4849a40f4d4504e37272', 'request_id': 'req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:50.301427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7459260f-67d7-4f3e-88ec-e369ef6c13bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:50.313150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8f2ac8c139b4849a40f4d4504e37272', 'request_id': 'req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:50.323767 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8f2ac8c139b4849a40f4d4504e37272', 'request_id': 'req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:50.341884 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e admin admin] The token cache is being invalidate because project a8f2ac8c139b4849a40f4d4504e37272 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:50.342707 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:50.343453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:50.345348 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 932/1870] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:50 2022] DELETE /identity/v3/projects/a8f2ac8c139b4849a40f4d4504e37272 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:50.349063 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:50.350463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:50.378309 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:50.394751 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 939/1871] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:50.406570 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459260f-67d7-4f3e-88ec-e369ef6c13bd None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:50.409058 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7459260f-67d7-4f3e-88ec-e369ef6c13bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:50.425842 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dddf6d2-c6a3-43e5-9cc0-4ad221a18a9e admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:50.432429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f62842-fe10-43c3-a335-340c2adf5f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:50.433193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f62842-fe10-43c3-a335-340c2adf5f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:50.434318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f62842-fe10-43c3-a335-340c2adf5f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:50.434318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f62842-fe10-43c3-a335-340c2adf5f3b admin admin] PATH_INFO: `/v3/users/56b423366d844479909fd50981a19334` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:50.435236 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f62842-fe10-43c3-a335-340c2adf5f3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b423366d844479909fd50981a19334)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:50.437460 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f62842-fe10-43c3-a335-340c2adf5f3b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:50.476138 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459260f-67d7-4f3e-88ec-e369ef6c13bd None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:50.481632 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4204c4-120b-4761-88e1-27e87c19f635 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:50.482267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4204c4-120b-4761-88e1-27e87c19f635 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:50.482549 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4204c4-120b-4761-88e1-27e87c19f635 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:50.482821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4204c4-120b-4761-88e1-27e87c19f635 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:50.484612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1c4204c4-120b-4761-88e1-27e87c19f635 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:50.488685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-32f62842-fe10-43c3-a335-340c2adf5f3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b423366d844479909fd50981a19334'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:50.489268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-32f62842-fe10-43c3-a335-340c2adf5f3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b423366d844479909fd50981a19334'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:50.489959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-32f62842-fe10-43c3-a335-340c2adf5f3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b423366d844479909fd50981a19334'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:50.508072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-32f62842-fe10-43c3-a335-340c2adf5f3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b423366d844479909fd50981a19334'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:50.516129 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-32f62842-fe10-43c3-a335-340c2adf5f3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b423366d844479909fd50981a19334'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:50.532707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:50.535584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 933/1872] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:50 2022] DELETE /identity/v3/users/56b423366d844479909fd50981a19334 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:50.568265 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4204c4-120b-4761-88e1-27e87c19f635 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:50.569713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4204c4-120b-4761-88e1-27e87c19f635 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:50.580020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 940/1873] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:17:50.984363 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f62842-fe10-43c3-a335-340c2adf5f3b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:50.986781 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-32f62842-fe10-43c3-a335-340c2adf5f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:51.191284 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f62842-fe10-43c3-a335-340c2adf5f3b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:51.198644 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832be8cd-868a-419c-9525-a30c6f06fac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:51.199588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832be8cd-868a-419c-9525-a30c6f06fac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:51.200046 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832be8cd-868a-419c-9525-a30c6f06fac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:51.200501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832be8cd-868a-419c-9525-a30c6f06fac2 admin admin] PATH_INFO: `/v3/projects/094143dbf2b048b0bc0b32af456608c4` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:51.202152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832be8cd-868a-419c-9525-a30c6f06fac2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=094143dbf2b048b0bc0b32af456608c4)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:51.204409 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832be8cd-868a-419c-9525-a30c6f06fac2 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:51.280060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:51.291199 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-832be8cd-868a-419c-9525-a30c6f06fac2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '094143dbf2b048b0bc0b32af456608c4', 'request_id': 'req-832be8cd-868a-419c-9525-a30c6f06fac2'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:51.302874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-832be8cd-868a-419c-9525-a30c6f06fac2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '094143dbf2b048b0bc0b32af456608c4', 'request_id': 'req-832be8cd-868a-419c-9525-a30c6f06fac2'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:51.312120 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-832be8cd-868a-419c-9525-a30c6f06fac2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '094143dbf2b048b0bc0b32af456608c4', 'request_id': 'req-832be8cd-868a-419c-9525-a30c6f06fac2'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:51.335541 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-832be8cd-868a-419c-9525-a30c6f06fac2 admin admin] The token cache is being invalidate because project 094143dbf2b048b0bc0b32af456608c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:51.336367 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-832be8cd-868a-419c-9525-a30c6f06fac2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:51.337246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-832be8cd-868a-419c-9525-a30c6f06fac2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:51.339405 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 934/1874] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:50 2022] DELETE /identity/v3/projects/094143dbf2b048b0bc0b32af456608c4 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:51.344341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0620272-a2db-41fa-9f2c-8981c49c2fa3 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:51.344341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0620272-a2db-41fa-9f2c-8981c49c2fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:51.344341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0620272-a2db-41fa-9f2c-8981c49c2fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:51.372930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-b0620272-a2db-41fa-9f2c-8981c49c2fa3 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:51.374568 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b0620272-a2db-41fa-9f2c-8981c49c2fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:51.431824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832be8cd-868a-419c-9525-a30c6f06fac2 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:51.433767 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-832be8cd-868a-419c-9525-a30c6f06fac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:51.470465 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 941/1875] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:51.480572 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd457bc-e7ad-43ba-ac10-53d393e3940a None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:51.480846 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd457bc-e7ad-43ba-ac10-53d393e3940a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:51.481122 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd457bc-e7ad-43ba-ac10-53d393e3940a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:51.504922 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-1fd457bc-e7ad-43ba-ac10-53d393e3940a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:51.506228 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1fd457bc-e7ad-43ba-ac10-53d393e3940a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:51.517454 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832be8cd-868a-419c-9525-a30c6f06fac2 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:51.519168 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 942/1876] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:51.520487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd457bc-e7ad-43ba-ac10-53d393e3940a None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:51.525014 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41287f-ae4d-4744-9682-bfd2a5e8dae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:51.525816 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41287f-ae4d-4744-9682-bfd2a5e8dae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:51.526201 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41287f-ae4d-4744-9682-bfd2a5e8dae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:51.526550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41287f-ae4d-4744-9682-bfd2a5e8dae5 admin admin] PATH_INFO: `/v3/users/70337b7913654cfdbb929ca4457aef21` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:51.532483 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd457bc-e7ad-43ba-ac10-53d393e3940a None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:51.541677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41287f-ae4d-4744-9682-bfd2a5e8dae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70337b7913654cfdbb929ca4457aef21)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:51.543799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41287f-ae4d-4744-9682-bfd2a5e8dae5 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:51.547440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5aa535-2d26-4acb-80d5-65f7f7507f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:51.548511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5aa535-2d26-4acb-80d5-65f7f7507f66 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:51.549340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5aa535-2d26-4acb-80d5-65f7f7507f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:51.549806 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5aa535-2d26-4acb-80d5-65f7f7507f66 admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:51.551051 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5aa535-2d26-4acb-80d5-65f7f7507f66 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:17:51.552372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5aa535-2d26-4acb-80d5-65f7f7507f66 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:51.554298 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5aa535-2d26-4acb-80d5-65f7f7507f66 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:51.591681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 943/1877] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:17:51.596046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5aa535-2d26-4acb-80d5-65f7f7507f66 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:51.610374 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5aa535-2d26-4acb-80d5-65f7f7507f66 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:51.615345 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44036c0-0ebc-45b7-b3f2-9b940fb686f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:51.616040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44036c0-0ebc-45b7-b3f2-9b940fb686f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:51.616399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44036c0-0ebc-45b7-b3f2-9b940fb686f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:51.616793 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44036c0-0ebc-45b7-b3f2-9b940fb686f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:51.617540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44036c0-0ebc-45b7-b3f2-9b940fb686f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:51.620463 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44036c0-0ebc-45b7-b3f2-9b940fb686f5 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:51.622283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4e41287f-ae4d-4744-9682-bfd2a5e8dae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70337b7913654cfdbb929ca4457aef21'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:51.623156 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4e41287f-ae4d-4744-9682-bfd2a5e8dae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70337b7913654cfdbb929ca4457aef21'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:51.623854 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4e41287f-ae4d-4744-9682-bfd2a5e8dae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70337b7913654cfdbb929ca4457aef21'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:51.639761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4e41287f-ae4d-4744-9682-bfd2a5e8dae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70337b7913654cfdbb929ca4457aef21'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:51.651978 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4e41287f-ae4d-4744-9682-bfd2a5e8dae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70337b7913654cfdbb929ca4457aef21'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:51.664653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 944/1878] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:51 2022] POST /identity/v3/projects => generated 387 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:17:51.666875 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44036c0-0ebc-45b7-b3f2-9b940fb686f5 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:51.668612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c44036c0-0ebc-45b7-b3f2-9b940fb686f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:51.676697 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 935/1879] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:51 2022] DELETE /identity/v3/users/70337b7913654cfdbb929ca4457aef21 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:51.683610 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0209d-c0ca-433d-9350-5dfd0d157528 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:51.683610 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0209d-c0ca-433d-9350-5dfd0d157528 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:51.683610 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0209d-c0ca-433d-9350-5dfd0d157528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:51.712415 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-85e0209d-c0ca-433d-9350-5dfd0d157528 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:51.714144 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-85e0209d-c0ca-433d-9350-5dfd0d157528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:51.770173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:51.777631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44036c0-0ebc-45b7-b3f2-9b940fb686f5 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:51.796752 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 936/1880] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:51.810531 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0209d-c0ca-433d-9350-5dfd0d157528 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:51.812878 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-85e0209d-c0ca-433d-9350-5dfd0d157528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:51.823030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:51.859860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eebc92-2905-4819-b060-c2e2fe461e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:51.860850 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eebc92-2905-4819-b060-c2e2fe461e6c admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:51.861627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eebc92-2905-4819-b060-c2e2fe461e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:51.862065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eebc92-2905-4819-b060-c2e2fe461e6c admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:51.863409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eebc92-2905-4819-b060-c2e2fe461e6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:51.867614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eebc92-2905-4819-b060-c2e2fe461e6c admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:51.902099 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 945/1881] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:51 2022] POST /identity/v3/users => generated 294 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:17:51.906654 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eebc92-2905-4819-b060-c2e2fe461e6c admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:51.929497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eebc92-2905-4819-b060-c2e2fe461e6c admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:51.935598 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0209d-c0ca-433d-9350-5dfd0d157528 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:51.941227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d84dd-c1a3-4153-aa22-db63aadf2a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:51.942326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d84dd-c1a3-4153-aa22-db63aadf2a89 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:51.943981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d84dd-c1a3-4153-aa22-db63aadf2a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:51.943981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d84dd-c1a3-4153-aa22-db63aadf2a89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:51.944867 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8c0c5-fd44-4c28-88f0-54fd43ae5d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:51.945949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d84dd-c1a3-4153-aa22-db63aadf2a89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:51.946686 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8c0c5-fd44-4c28-88f0-54fd43ae5d66 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:51.948307 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8c0c5-fd44-4c28-88f0-54fd43ae5d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:51.949132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d84dd-c1a3-4153-aa22-db63aadf2a89 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:51.949247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8c0c5-fd44-4c28-88f0-54fd43ae5d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:51.961456 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8c0c5-fd44-4c28-88f0-54fd43ae5d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:51.966801 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8c0c5-fd44-4c28-88f0-54fd43ae5d66 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:51.971515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 946/1882] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:51.975538 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d84dd-c1a3-4153-aa22-db63aadf2a89 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:51.977961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 937/1883] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:17:51.986197 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d84dd-c1a3-4153-aa22-db63aadf2a89 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:51.990836 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e513d27-4039-4295-9045-6273db5b6ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:51.991442 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e513d27-4039-4295-9045-6273db5b6ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:51.991722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e513d27-4039-4295-9045-6273db5b6ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:51.992111 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e513d27-4039-4295-9045-6273db5b6ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:51.993202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e513d27-4039-4295-9045-6273db5b6ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:51.994818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e513d27-4039-4295-9045-6273db5b6ea5 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:52.007375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 947/1884] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:17:52.012190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8c0c5-fd44-4c28-88f0-54fd43ae5d66 service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:52.025351 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8c0c5-fd44-4c28-88f0-54fd43ae5d66 service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:52.030547 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1930f8-01f1-4e3f-b275-92e5519becc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:52.031223 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1930f8-01f1-4e3f-b275-92e5519becc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:52.031499 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1930f8-01f1-4e3f-b275-92e5519becc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:52.031899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1930f8-01f1-4e3f-b275-92e5519becc3 admin admin] PATH_INFO: `/v3/projects/2662f0954b444b13bc3a8d06010f3675/users/c7b28280de4547daad59da2d894a9eef/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:52.048389 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1930f8-01f1-4e3f-b275-92e5519becc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2662f0954b444b13bc3a8d06010f3675, user_id=c7b28280de4547daad59da2d894a9eef, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:52.051202 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1930f8-01f1-4e3f-b275-92e5519becc3 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:52.069385 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 938/1885] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:52 2022] PUT /identity/v3/projects/2662f0954b444b13bc3a8d06010f3675/users/c7b28280de4547daad59da2d894a9eef/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:52.080930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946577b8-0a40-4deb-b374-1c7208425b0d None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:52.081254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946577b8-0a40-4deb-b374-1c7208425b0d None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:52.081558 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946577b8-0a40-4deb-b374-1c7208425b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:52.122775 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-946577b8-0a40-4deb-b374-1c7208425b0d None None] MFA Rules not processed for user `c7b28280de4547daad59da2d894a9eef`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:52.124189 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-946577b8-0a40-4deb-b374-1c7208425b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:52.250599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 948/1886] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:52.255301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1930f8-01f1-4e3f-b275-92e5519becc3 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:52.340473 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1930f8-01f1-4e3f-b275-92e5519becc3 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:52.347032 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd71b7-63c6-4458-aa76-fc1e92a75bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:52.347680 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd71b7-63c6-4458-aa76-fc1e92a75bac admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:52.347978 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd71b7-63c6-4458-aa76-fc1e92a75bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:52.348235 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd71b7-63c6-4458-aa76-fc1e92a75bac admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:52.348954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd71b7-63c6-4458-aa76-fc1e92a75bac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:52.351096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd71b7-63c6-4458-aa76-fc1e92a75bac admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:52.403140 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 939/1887] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:52 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:17:52.404495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946577b8-0a40-4deb-b374-1c7208425b0d None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:52.425448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946577b8-0a40-4deb-b374-1c7208425b0d None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:52.459226 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd71b7-63c6-4458-aa76-fc1e92a75bac admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:52.462514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-29bd71b7-63c6-4458-aa76-fc1e92a75bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:52.513183 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f410512-6583-4527-9d41-52571ceee582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:52.513639 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f410512-6583-4527-9d41-52571ceee582 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:52.513879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f410512-6583-4527-9d41-52571ceee582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:52.514150 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f410512-6583-4527-9d41-52571ceee582 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:52.515111 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f410512-6583-4527-9d41-52571ceee582 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:52.517322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f410512-6583-4527-9d41-52571ceee582 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:52.531651 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd71b7-63c6-4458-aa76-fc1e92a75bac admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:52.536550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffcd8c-ccbc-4985-8f3a-bc9b8566a984 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:52.537445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffcd8c-ccbc-4985-8f3a-bc9b8566a984 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:52.537885 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffcd8c-ccbc-4985-8f3a-bc9b8566a984 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:52.538312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffcd8c-ccbc-4985-8f3a-bc9b8566a984 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:52.540762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d7ffcd8c-ccbc-4985-8f3a-bc9b8566a984 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:52.546997 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 949/1888] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:52 2022] POST /identity/v3/users => generated 300 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:52.552233 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f410512-6583-4527-9d41-52571ceee582 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:52.558052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:52.563913 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f410512-6583-4527-9d41-52571ceee582 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:52.568938 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2951543-3d20-4ee1-b5c0-c8d45b64d561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:52.569649 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2951543-3d20-4ee1-b5c0-c8d45b64d561 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:52.570510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2951543-3d20-4ee1-b5c0-c8d45b64d561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:52.570854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2951543-3d20-4ee1-b5c0-c8d45b64d561 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:52.572020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2951543-3d20-4ee1-b5c0-c8d45b64d561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:52.573666 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2951543-3d20-4ee1-b5c0-c8d45b64d561 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:52.607119 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 950/1889] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:52.608132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2951543-3d20-4ee1-b5c0-c8d45b64d561 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:52.619488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ffcd8c-ccbc-4985-8f3a-bc9b8566a984 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:52.620439 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2951543-3d20-4ee1-b5c0-c8d45b64d561 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:52.621234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ffcd8c-ccbc-4985-8f3a-bc9b8566a984 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:52.626525 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c398dd-e764-4151-9d23-be4c950073a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:52.627492 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c398dd-e764-4151-9d23-be4c950073a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:52.627978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c398dd-e764-4151-9d23-be4c950073a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:52.628351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c398dd-e764-4151-9d23-be4c950073a7 admin admin] PATH_INFO: `/v3/projects/7ec9eb9ae96740ca9312a17977f01d7f/users/a405e47f9cb145dbbf0e3a1d225756b5/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:52.643225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:52.649673 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c398dd-e764-4151-9d23-be4c950073a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec9eb9ae96740ca9312a17977f01d7f, user_id=a405e47f9cb145dbbf0e3a1d225756b5, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:52.650515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 940/1890] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:52.655979 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c398dd-e764-4151-9d23-be4c950073a7 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:52.711418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 951/1891] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:52 2022] PUT /identity/v3/projects/7ec9eb9ae96740ca9312a17977f01d7f/users/a405e47f9cb145dbbf0e3a1d225756b5/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:52.713001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffcd8c-ccbc-4985-8f3a-bc9b8566a984 service cinder] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:52.724869 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffcd8c-ccbc-4985-8f3a-bc9b8566a984 service cinder] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:52.844877 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba3f17e-eb12-45d9-8843-4ef5e72e38df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:52.845531 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3f17e-eb12-45d9-8843-4ef5e72e38df admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:52.845795 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3f17e-eb12-45d9-8843-4ef5e72e38df admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:52.846081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba3f17e-eb12-45d9-8843-4ef5e72e38df admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:52.846794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3f17e-eb12-45d9-8843-4ef5e72e38df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:52.848723 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba3f17e-eb12-45d9-8843-4ef5e72e38df admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:52.869014 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 941/1892] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:52.873533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c398dd-e764-4151-9d23-be4c950073a7 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:52.889418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c398dd-e764-4151-9d23-be4c950073a7 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:52.901624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b963bb3-a6af-4ddc-8a2d-7668731b9b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:52.903104 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b963bb3-a6af-4ddc-8a2d-7668731b9b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:52.903468 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b963bb3-a6af-4ddc-8a2d-7668731b9b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:52.903816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b963bb3-a6af-4ddc-8a2d-7668731b9b99 admin admin] PATH_INFO: `/v3/projects/7ec9eb9ae96740ca9312a17977f01d7f/users/a405e47f9cb145dbbf0e3a1d225756b5/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:52.908600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b963bb3-a6af-4ddc-8a2d-7668731b9b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec9eb9ae96740ca9312a17977f01d7f, user_id=a405e47f9cb145dbbf0e3a1d225756b5, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:52.912897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b963bb3-a6af-4ddc-8a2d-7668731b9b99 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:52.938439 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 952/1893] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:52 2022] PUT /identity/v3/projects/7ec9eb9ae96740ca9312a17977f01d7f/users/a405e47f9cb145dbbf0e3a1d225756b5/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:52.948475 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea50e1-7d35-42de-af72-d582d280c3b3 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:52.949036 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea50e1-7d35-42de-af72-d582d280c3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:52.949481 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea50e1-7d35-42de-af72-d582d280c3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:52.991817 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-beea50e1-7d35-42de-af72-d582d280c3b3 None None] MFA Rules not processed for user `a405e47f9cb145dbbf0e3a1d225756b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:52.993387 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-beea50e1-7d35-42de-af72-d582d280c3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:53.112687 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 942/1894] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:52 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:53.123162 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b963bb3-a6af-4ddc-8a2d-7668731b9b99 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:53.125207 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8b963bb3-a6af-4ddc-8a2d-7668731b9b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:53.200236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b963bb3-a6af-4ddc-8a2d-7668731b9b99 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:53.204321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1aa6ce-1db0-4e84-af75-081b04f38c05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:53.205386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1aa6ce-1db0-4e84-af75-081b04f38c05 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:53.205667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1aa6ce-1db0-4e84-af75-081b04f38c05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:53.205937 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1aa6ce-1db0-4e84-af75-081b04f38c05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:53.224649 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1aa6ce-1db0-4e84-af75-081b04f38c05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:53.231100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1aa6ce-1db0-4e84-af75-081b04f38c05 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:53.248825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 953/1895] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:53 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:53.277435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea50e1-7d35-42de-af72-d582d280c3b3 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:53.278979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-beea50e1-7d35-42de-af72-d582d280c3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:53.363992 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:53.432173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea50e1-7d35-42de-af72-d582d280c3b3 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:53.442793 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:53.443677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:53.443963 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:53.445974 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e admin admin] PATH_INFO: `/v3/projects/e5ee77f9c0594d7990feee496314e3f9` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:53.447583 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5ee77f9c0594d7990feee496314e3f9)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:53.449565 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:53.543895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5ee77f9c0594d7990feee496314e3f9', 'request_id': 'req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:53.575405 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5ee77f9c0594d7990feee496314e3f9', 'request_id': 'req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:53.582731 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5ee77f9c0594d7990feee496314e3f9', 'request_id': 'req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:53.596743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e admin admin] The token cache is being invalidate because project e5ee77f9c0594d7990feee496314e3f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:53.597155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:53.598682 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:53.603833 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1aa6ce-1db0-4e84-af75-081b04f38c05 service cinder] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:53.608702 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 943/1896] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:53 2022] DELETE /identity/v3/projects/e5ee77f9c0594d7990feee496314e3f9 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:53.625959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:53.626333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1aa6ce-1db0-4e84-af75-081b04f38c05 service cinder] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:53.627591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:53.698163 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f41fd5-4636-44df-bfe1-1ee1dd2199ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:53.698967 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f41fd5-4636-44df-bfe1-1ee1dd2199ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:53.699777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f41fd5-4636-44df-bfe1-1ee1dd2199ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:53.700164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f41fd5-4636-44df-bfe1-1ee1dd2199ea admin admin] PATH_INFO: `/v3/users/02e8736d24cb44f3af9631ee40bed495` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:53.701629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f41fd5-4636-44df-bfe1-1ee1dd2199ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=02e8736d24cb44f3af9631ee40bed495)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:53.704046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f41fd5-4636-44df-bfe1-1ee1dd2199ea admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:53.782437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9a2de-bbd6-45e1-84eb-c9975a25425e admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:53.793001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a648ba-5c67-4233-9cae-7e8d43a5ad91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:53.793642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a648ba-5c67-4233-9cae-7e8d43a5ad91 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:53.793952 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a648ba-5c67-4233-9cae-7e8d43a5ad91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:53.794179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a648ba-5c67-4233-9cae-7e8d43a5ad91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:53.796007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-58a648ba-5c67-4233-9cae-7e8d43a5ad91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:53.796128 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-21f41fd5-4636-44df-bfe1-1ee1dd2199ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02e8736d24cb44f3af9631ee40bed495'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:53.807087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-21f41fd5-4636-44df-bfe1-1ee1dd2199ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02e8736d24cb44f3af9631ee40bed495'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:53.808147 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-21f41fd5-4636-44df-bfe1-1ee1dd2199ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02e8736d24cb44f3af9631ee40bed495'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:53.831895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-21f41fd5-4636-44df-bfe1-1ee1dd2199ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02e8736d24cb44f3af9631ee40bed495'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:53.833424 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-21f41fd5-4636-44df-bfe1-1ee1dd2199ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02e8736d24cb44f3af9631ee40bed495'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:53.848665 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:53.862050 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 954/1897] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:53 2022] DELETE /identity/v3/users/02e8736d24cb44f3af9631ee40bed495 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:53.884668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a648ba-5c67-4233-9cae-7e8d43a5ad91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:53.886094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a648ba-5c67-4233-9cae-7e8d43a5ad91 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:53.902013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 944/1898] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:54.297572 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f41fd5-4636-44df-bfe1-1ee1dd2199ea admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:54.300945 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-21f41fd5-4636-44df-bfe1-1ee1dd2199ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:54.315081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a648ba-5c67-4233-9cae-7e8d43a5ad91 service cinder] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:54.317493 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-58a648ba-5c67-4233-9cae-7e8d43a5ad91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:54.354549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a648ba-5c67-4233-9cae-7e8d43a5ad91 service cinder] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:54.366404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:54.463518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f41fd5-4636-44df-bfe1-1ee1dd2199ea admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:54.469912 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08279550-63fc-4389-97a4-8967bee2fb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:54.470756 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08279550-63fc-4389-97a4-8967bee2fb5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:54.471098 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08279550-63fc-4389-97a4-8967bee2fb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:54.471671 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08279550-63fc-4389-97a4-8967bee2fb5d admin admin] PATH_INFO: `/v3/projects/d72cb0368e5a4522975f61b451fb0aad` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:54.472769 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08279550-63fc-4389-97a4-8967bee2fb5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d72cb0368e5a4522975f61b451fb0aad)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:54.474716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08279550-63fc-4389-97a4-8967bee2fb5d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:54.501690 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6079d-966d-432b-aeb5-2842bf98e661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:54.505396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6079d-966d-432b-aeb5-2842bf98e661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:54.505776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6079d-966d-432b-aeb5-2842bf98e661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:54.506064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6079d-966d-432b-aeb5-2842bf98e661 admin admin] PATH_INFO: `/v3/users/c7b28280de4547daad59da2d894a9eef` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:54.514548 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6079d-966d-432b-aeb5-2842bf98e661 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7b28280de4547daad59da2d894a9eef)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:54.518927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6079d-966d-432b-aeb5-2842bf98e661 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:54.583888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-08279550-63fc-4389-97a4-8967bee2fb5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd72cb0368e5a4522975f61b451fb0aad', 'request_id': 'req-08279550-63fc-4389-97a4-8967bee2fb5d'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:54.608595 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-98f6079d-966d-432b-aeb5-2842bf98e661 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7b28280de4547daad59da2d894a9eef'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:54.609940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-98f6079d-966d-432b-aeb5-2842bf98e661 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7b28280de4547daad59da2d894a9eef'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:54.610744 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-98f6079d-966d-432b-aeb5-2842bf98e661 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7b28280de4547daad59da2d894a9eef'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:54.614617 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-08279550-63fc-4389-97a4-8967bee2fb5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd72cb0368e5a4522975f61b451fb0aad', 'request_id': 'req-08279550-63fc-4389-97a4-8967bee2fb5d'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:54.623379 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-08279550-63fc-4389-97a4-8967bee2fb5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd72cb0368e5a4522975f61b451fb0aad', 'request_id': 'req-08279550-63fc-4389-97a4-8967bee2fb5d'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:54.630033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-98f6079d-966d-432b-aeb5-2842bf98e661 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7b28280de4547daad59da2d894a9eef'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:54.638317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-98f6079d-966d-432b-aeb5-2842bf98e661 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7b28280de4547daad59da2d894a9eef'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:54.639171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-08279550-63fc-4389-97a4-8967bee2fb5d admin admin] The token cache is being invalidate because project d72cb0368e5a4522975f61b451fb0aad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:54.639475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-08279550-63fc-4389-97a4-8967bee2fb5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:54.640160 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-08279550-63fc-4389-97a4-8967bee2fb5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:54.645714 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 955/1899] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:54 2022] DELETE /identity/v3/projects/d72cb0368e5a4522975f61b451fb0aad => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:54.649420 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d62e6-97a3-456b-85e2-764171894f06 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:54.649735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d62e6-97a3-456b-85e2-764171894f06 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:54.650355 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d62e6-97a3-456b-85e2-764171894f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:54.675108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 945/1900] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:54 2022] DELETE /identity/v3/users/c7b28280de4547daad59da2d894a9eef => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:54.681920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5de3b3-42b0-4a89-bda9-0db397aa2a93 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:54.682367 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5de3b3-42b0-4a89-bda9-0db397aa2a93 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:54.682905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5de3b3-42b0-4a89-bda9-0db397aa2a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:54.700218 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-346d62e6-97a3-456b-85e2-764171894f06 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:54.702398 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-346d62e6-97a3-456b-85e2-764171894f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:54.744164 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-cb5de3b3-42b0-4a89-bda9-0db397aa2a93 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:54.748046 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-cb5de3b3-42b0-4a89-bda9-0db397aa2a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:54.780380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 956/1901] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:54.782679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e86c5f-1c67-49f1-a424-9c35b3e483c7 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:54.782964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e86c5f-1c67-49f1-a424-9c35b3e483c7 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:54.783251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e86c5f-1c67-49f1-a424-9c35b3e483c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:54.817959 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-50e86c5f-1c67-49f1-a424-9c35b3e483c7 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:54.821134 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-50e86c5f-1c67-49f1-a424-9c35b3e483c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:54.834388 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 957/1902] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:54.839132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e86c5f-1c67-49f1-a424-9c35b3e483c7 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:54.859138 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e86c5f-1c67-49f1-a424-9c35b3e483c7 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:54.868909 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b3d8a-61b8-4ef2-9379-d1a6e98a942e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:54.869904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b3d8a-61b8-4ef2-9379-d1a6e98a942e admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:54.870489 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b3d8a-61b8-4ef2-9379-d1a6e98a942e admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:54.870949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983b3d8a-61b8-4ef2-9379-d1a6e98a942e admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:54.871844 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b3d8a-61b8-4ef2-9379-d1a6e98a942e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:17:54.872443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b3d8a-61b8-4ef2-9379-d1a6e98a942e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:54.874496 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983b3d8a-61b8-4ef2-9379-d1a6e98a942e admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:54.887991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 946/1903] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:54.895074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 958/1904] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:17:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:17:54.896513 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5de3b3-42b0-4a89-bda9-0db397aa2a93 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:54.901666 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b3d8a-61b8-4ef2-9379-d1a6e98a942e admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:54.903921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-983b3d8a-61b8-4ef2-9379-d1a6e98a942e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:54.908388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5de3b3-42b0-4a89-bda9-0db397aa2a93 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:54.913661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc281a-71a4-44db-8823-c9b5bc5039c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:54.914571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc281a-71a4-44db-8823-c9b5bc5039c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:54.915069 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc281a-71a4-44db-8823-c9b5bc5039c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:54.915491 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc281a-71a4-44db-8823-c9b5bc5039c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:54.916325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc281a-71a4-44db-8823-c9b5bc5039c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:54.918415 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:54.918656 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc281a-71a4-44db-8823-c9b5bc5039c1 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:54.954827 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 947/1905] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:54 2022] POST /identity/v3/projects => generated 363 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:54.960351 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc281a-71a4-44db-8823-c9b5bc5039c1 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:54.983938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:55.003769 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983b3d8a-61b8-4ef2-9379-d1a6e98a942e admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:55.075036 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc281a-71a4-44db-8823-c9b5bc5039c1 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:55.083759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46de28-944c-4dc4-9bfd-bdfc00fa7fed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:55.084433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46de28-944c-4dc4-9bfd-bdfc00fa7fed service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:55.084745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46de28-944c-4dc4-9bfd-bdfc00fa7fed service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:55.085117 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a46de28-944c-4dc4-9bfd-bdfc00fa7fed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:55.087877 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731f620-3db7-4a10-beac-65dfcb292f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:55.088636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731f620-3db7-4a10-beac-65dfcb292f79 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:55.089074 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731f620-3db7-4a10-beac-65dfcb292f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:55.089467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731f620-3db7-4a10-beac-65dfcb292f79 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:55.090236 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731f620-3db7-4a10-beac-65dfcb292f79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:55.092264 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731f620-3db7-4a10-beac-65dfcb292f79 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:55.096947 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46de28-944c-4dc4-9bfd-bdfc00fa7fed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:55.098919 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a46de28-944c-4dc4-9bfd-bdfc00fa7fed service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:55.120125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 948/1906] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:54 2022] POST /identity/v3/users => generated 282 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:55.121404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731f620-3db7-4a10-beac-65dfcb292f79 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:55.125602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 959/1907] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:55.133834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731f620-3db7-4a10-beac-65dfcb292f79 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:55.139833 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c1f57-2db7-4fc6-91b6-57957504666f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:55.141373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c1f57-2db7-4fc6-91b6-57957504666f admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:55.141758 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c1f57-2db7-4fc6-91b6-57957504666f admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:55.142134 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c1f57-2db7-4fc6-91b6-57957504666f admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:55.142943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c1f57-2db7-4fc6-91b6-57957504666f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:55.146364 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c1f57-2db7-4fc6-91b6-57957504666f admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:55.158605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 949/1908] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:55.163017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46de28-944c-4dc4-9bfd-bdfc00fa7fed service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:55.181555 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a46de28-944c-4dc4-9bfd-bdfc00fa7fed service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:55.187843 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce77dc6-2aef-4cfc-8ac1-c01404fa84cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:55.189518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce77dc6-2aef-4cfc-8ac1-c01404fa84cb admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:55.189915 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce77dc6-2aef-4cfc-8ac1-c01404fa84cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:55.191228 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce77dc6-2aef-4cfc-8ac1-c01404fa84cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:55.192791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce77dc6-2aef-4cfc-8ac1-c01404fa84cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:55.194552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce77dc6-2aef-4cfc-8ac1-c01404fa84cb admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:55.213367 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 960/1909] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:55.216973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c1f57-2db7-4fc6-91b6-57957504666f admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:55.231191 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c1f57-2db7-4fc6-91b6-57957504666f admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:55.244169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38886312-c22e-4fac-85ae-681bc2db63d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:55.245071 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38886312-c22e-4fac-85ae-681bc2db63d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:55.245658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38886312-c22e-4fac-85ae-681bc2db63d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:55.246284 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38886312-c22e-4fac-85ae-681bc2db63d9 admin admin] PATH_INFO: `/v3/projects/5bc9909ab42b4e578faa517387780320/users/21a0f892e60c463586c7d919eaaa8be0/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:55.281922 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38886312-c22e-4fac-85ae-681bc2db63d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc9909ab42b4e578faa517387780320, user_id=21a0f892e60c463586c7d919eaaa8be0, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:55.284689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38886312-c22e-4fac-85ae-681bc2db63d9 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:55.337775 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d6641-e27a-4e5c-b99d-262f1b7a6ad3 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:55.338329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d6641-e27a-4e5c-b99d-262f1b7a6ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:55.338690 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332d6641-e27a-4e5c-b99d-262f1b7a6ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:55.343781 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 950/1910] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:55 2022] PUT /identity/v3/projects/5bc9909ab42b4e578faa517387780320/users/21a0f892e60c463586c7d919eaaa8be0/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:55.405706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-332d6641-e27a-4e5c-b99d-262f1b7a6ad3 None None] MFA Rules not processed for user `21a0f892e60c463586c7d919eaaa8be0`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:55.406860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-332d6641-e27a-4e5c-b99d-262f1b7a6ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:55.556957 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38886312-c22e-4fac-85ae-681bc2db63d9 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:55.558864 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-38886312-c22e-4fac-85ae-681bc2db63d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:55.602652 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 961/1911] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:55.608221 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d6641-e27a-4e5c-b99d-262f1b7a6ad3 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:55.722537 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38886312-c22e-4fac-85ae-681bc2db63d9 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:55.738581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137bb1e2-0dd3-441b-9931-1ca47895641a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:55.739630 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137bb1e2-0dd3-441b-9931-1ca47895641a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:55.740071 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137bb1e2-0dd3-441b-9931-1ca47895641a admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:55.740396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137bb1e2-0dd3-441b-9931-1ca47895641a admin admin] PATH_INFO: `/v3/projects/2662f0954b444b13bc3a8d06010f3675` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:55.747869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:55.758272 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137bb1e2-0dd3-441b-9931-1ca47895641a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2662f0954b444b13bc3a8d06010f3675)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:55.764685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137bb1e2-0dd3-441b-9931-1ca47895641a admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:55.793063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332d6641-e27a-4e5c-b99d-262f1b7a6ad3 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:55.799258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b89796-0e7e-4ddf-9825-0d7ccc94e42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:55.800329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b89796-0e7e-4ddf-9825-0d7ccc94e42d admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:55.800640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b89796-0e7e-4ddf-9825-0d7ccc94e42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:55.801094 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b89796-0e7e-4ddf-9825-0d7ccc94e42d admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:55.801843 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b89796-0e7e-4ddf-9825-0d7ccc94e42d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:55.805001 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b89796-0e7e-4ddf-9825-0d7ccc94e42d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:55.881676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 962/1912] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:55 2022] POST /identity/v3/projects => generated 365 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:55.885617 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b89796-0e7e-4ddf-9825-0d7ccc94e42d admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:55.893889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-137bb1e2-0dd3-441b-9931-1ca47895641a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2662f0954b444b13bc3a8d06010f3675', 'request_id': 'req-137bb1e2-0dd3-441b-9931-1ca47895641a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:55.912997 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-137bb1e2-0dd3-441b-9931-1ca47895641a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2662f0954b444b13bc3a8d06010f3675', 'request_id': 'req-137bb1e2-0dd3-441b-9931-1ca47895641a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:55.932354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:55.937014 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-137bb1e2-0dd3-441b-9931-1ca47895641a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2662f0954b444b13bc3a8d06010f3675', 'request_id': 'req-137bb1e2-0dd3-441b-9931-1ca47895641a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:55.954332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-137bb1e2-0dd3-441b-9931-1ca47895641a admin admin] The token cache is being invalidate because project 2662f0954b444b13bc3a8d06010f3675 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:55.954800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-137bb1e2-0dd3-441b-9931-1ca47895641a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:55.955731 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-137bb1e2-0dd3-441b-9931-1ca47895641a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:55.957922 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 951/1913] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:55 2022] DELETE /identity/v3/projects/2662f0954b444b13bc3a8d06010f3675 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:55.962735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137bb1e2-0dd3-441b-9931-1ca47895641a admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:55.965104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-137bb1e2-0dd3-441b-9931-1ca47895641a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:56.005197 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b89796-0e7e-4ddf-9825-0d7ccc94e42d admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:56.012513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96cfda-b1ee-40d9-80e9-0d21c284c598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:56.013761 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96cfda-b1ee-40d9-80e9-0d21c284c598 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:56.014340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96cfda-b1ee-40d9-80e9-0d21c284c598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:56.014765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96cfda-b1ee-40d9-80e9-0d21c284c598 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:56.015842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96cfda-b1ee-40d9-80e9-0d21c284c598 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:56.018091 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96cfda-b1ee-40d9-80e9-0d21c284c598 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:56.046278 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 963/1914] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:55 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:56.047373 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96cfda-b1ee-40d9-80e9-0d21c284c598 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:56.050795 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9d96cfda-b1ee-40d9-80e9-0d21c284c598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:56.058014 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137bb1e2-0dd3-441b-9931-1ca47895641a admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:56.066874 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96cfda-b1ee-40d9-80e9-0d21c284c598 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:56.072413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e208c8c-f3da-440c-bfc9-a680d3146e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:56.073017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e208c8c-f3da-440c-bfc9-a680d3146e80 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:56.073305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e208c8c-f3da-440c-bfc9-a680d3146e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:56.073574 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e208c8c-f3da-440c-bfc9-a680d3146e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:56.074282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e208c8c-f3da-440c-bfc9-a680d3146e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:56.076140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e208c8c-f3da-440c-bfc9-a680d3146e80 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:56.077310 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea0dd9-b12e-4df8-abfe-1ed4d7ead88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:56.078239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0dd9-b12e-4df8-abfe-1ed4d7ead88c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:56.079864 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0dd9-b12e-4df8-abfe-1ed4d7ead88c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:56.079864 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0dd9-b12e-4df8-abfe-1ed4d7ead88c admin admin] PATH_INFO: `/v3/users/a405e47f9cb145dbbf0e3a1d225756b5` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:56.081255 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea0dd9-b12e-4df8-abfe-1ed4d7ead88c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a405e47f9cb145dbbf0e3a1d225756b5)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:56.083879 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea0dd9-b12e-4df8-abfe-1ed4d7ead88c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:56.093316 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 964/1915] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:56.094231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e208c8c-f3da-440c-bfc9-a680d3146e80 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:56.120360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e208c8c-f3da-440c-bfc9-a680d3146e80 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:56.129813 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71658d9d-dfd3-4d1a-b3dc-c11c213bd09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:56.130453 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71658d9d-dfd3-4d1a-b3dc-c11c213bd09d admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:56.130743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71658d9d-dfd3-4d1a-b3dc-c11c213bd09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:56.131002 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71658d9d-dfd3-4d1a-b3dc-c11c213bd09d admin admin] PATH_INFO: `/v3/projects/ff9c9354e0164e55ae92308199240cb7/users/30e2a481134a494b8f2a8a5dc5433253/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:56.147873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71658d9d-dfd3-4d1a-b3dc-c11c213bd09d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff9c9354e0164e55ae92308199240cb7, user_id=30e2a481134a494b8f2a8a5dc5433253, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:56.161292 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71658d9d-dfd3-4d1a-b3dc-c11c213bd09d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:56.164648 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-91ea0dd9-b12e-4df8-abfe-1ed4d7ead88c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a405e47f9cb145dbbf0e3a1d225756b5'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:56.166375 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-91ea0dd9-b12e-4df8-abfe-1ed4d7ead88c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a405e47f9cb145dbbf0e3a1d225756b5'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:56.167168 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-91ea0dd9-b12e-4df8-abfe-1ed4d7ead88c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a405e47f9cb145dbbf0e3a1d225756b5'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:56.182866 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-91ea0dd9-b12e-4df8-abfe-1ed4d7ead88c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a405e47f9cb145dbbf0e3a1d225756b5'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:56.191510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-91ea0dd9-b12e-4df8-abfe-1ed4d7ead88c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a405e47f9cb145dbbf0e3a1d225756b5'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:56.192726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 965/1916] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:56 2022] PUT /identity/v3/projects/ff9c9354e0164e55ae92308199240cb7/users/30e2a481134a494b8f2a8a5dc5433253/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:56.199783 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71658d9d-dfd3-4d1a-b3dc-c11c213bd09d admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:56.203745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-71658d9d-dfd3-4d1a-b3dc-c11c213bd09d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:56.214205 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 952/1917] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:55 2022] DELETE /identity/v3/users/a405e47f9cb145dbbf0e3a1d225756b5 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:56.339129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71658d9d-dfd3-4d1a-b3dc-c11c213bd09d admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:56.409357 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:56.449830 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd5fa9-d3d7-490e-97d6-dbbe4ec95246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:56.450457 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd5fa9-d3d7-490e-97d6-dbbe4ec95246 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:56.450781 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd5fa9-d3d7-490e-97d6-dbbe4ec95246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:56.453214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd5fa9-d3d7-490e-97d6-dbbe4ec95246 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:56.454044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd5fa9-d3d7-490e-97d6-dbbe4ec95246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:56.459476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdd5fa9-d3d7-490e-97d6-dbbe4ec95246 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:56.485455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 966/1918] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:56 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:17:56.489231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea0dd9-b12e-4df8-abfe-1ed4d7ead88c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:56.502022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea0dd9-b12e-4df8-abfe-1ed4d7ead88c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:56.510408 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51eb26-665a-487d-99a7-40b7a150c5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:56.511395 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51eb26-665a-487d-99a7-40b7a150c5ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:56.511827 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51eb26-665a-487d-99a7-40b7a150c5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:56.512267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51eb26-665a-487d-99a7-40b7a150c5ea admin admin] PATH_INFO: `/v3/projects/ff9c9354e0164e55ae92308199240cb7/users/30e2a481134a494b8f2a8a5dc5433253/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:56.515711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51eb26-665a-487d-99a7-40b7a150c5ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff9c9354e0164e55ae92308199240cb7, user_id=30e2a481134a494b8f2a8a5dc5433253, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:56.519560 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51eb26-665a-487d-99a7-40b7a150c5ea admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:56.542088 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 953/1919] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:56 2022] PUT /identity/v3/projects/ff9c9354e0164e55ae92308199240cb7/users/30e2a481134a494b8f2a8a5dc5433253/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:56.556916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17c6c3-89a6-47e1-a4dd-334430e15b18 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:56.558290 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17c6c3-89a6-47e1-a4dd-334430e15b18 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:56.558658 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17c6c3-89a6-47e1-a4dd-334430e15b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:56.613805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-1a17c6c3-89a6-47e1-a4dd-334430e15b18 None None] MFA Rules not processed for user `30e2a481134a494b8f2a8a5dc5433253`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:56.615297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1a17c6c3-89a6-47e1-a4dd-334430e15b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:56.660777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51eb26-665a-487d-99a7-40b7a150c5ea admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:56.664022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7f51eb26-665a-487d-99a7-40b7a150c5ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:56.747619 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 967/1920] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:56 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:56.755452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17c6c3-89a6-47e1-a4dd-334430e15b18 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:56.757545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1a17c6c3-89a6-47e1-a4dd-334430e15b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:56.783913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:56.789922 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51eb26-665a-487d-99a7-40b7a150c5ea admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:56.812431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9298a-e5bd-437c-8db7-98a1d1994621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:56.816993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9298a-e5bd-437c-8db7-98a1d1994621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:56.817327 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9298a-e5bd-437c-8db7-98a1d1994621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:56.817495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9298a-e5bd-437c-8db7-98a1d1994621 admin admin] PATH_INFO: `/v3/projects/7ec9eb9ae96740ca9312a17977f01d7f` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:56.819174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9298a-e5bd-437c-8db7-98a1d1994621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec9eb9ae96740ca9312a17977f01d7f)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:56.821715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9298a-e5bd-437c-8db7-98a1d1994621 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:56.894149 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17c6c3-89a6-47e1-a4dd-334430e15b18 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:56.901671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-edb9298a-e5bd-437c-8db7-98a1d1994621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec9eb9ae96740ca9312a17977f01d7f', 'request_id': 'req-edb9298a-e5bd-437c-8db7-98a1d1994621'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:56.913572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-edb9298a-e5bd-437c-8db7-98a1d1994621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec9eb9ae96740ca9312a17977f01d7f', 'request_id': 'req-edb9298a-e5bd-437c-8db7-98a1d1994621'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:56.922865 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-edb9298a-e5bd-437c-8db7-98a1d1994621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec9eb9ae96740ca9312a17977f01d7f', 'request_id': 'req-edb9298a-e5bd-437c-8db7-98a1d1994621'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:56.957582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-edb9298a-e5bd-437c-8db7-98a1d1994621 admin admin] The token cache is being invalidate because project 7ec9eb9ae96740ca9312a17977f01d7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:56.958078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-edb9298a-e5bd-437c-8db7-98a1d1994621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:56.959662 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-edb9298a-e5bd-437c-8db7-98a1d1994621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:56.974660 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 954/1921] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:56 2022] DELETE /identity/v3/projects/7ec9eb9ae96740ca9312a17977f01d7f => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:56.980467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdd5ba-49c6-440f-bfd8-ca6ad17d876e None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:56.980467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdd5ba-49c6-440f-bfd8-ca6ad17d876e None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:56.980942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdd5ba-49c6-440f-bfd8-ca6ad17d876e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:56.988404 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebb2fd-d0c7-4d3b-88b5-5d4628e042df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:56.989118 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebb2fd-d0c7-4d3b-88b5-5d4628e042df service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:56.989477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebb2fd-d0c7-4d3b-88b5-5d4628e042df service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:56.989759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebb2fd-d0c7-4d3b-88b5-5d4628e042df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:56.991617 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-ccebb2fd-d0c7-4d3b-88b5-5d4628e042df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:57.024185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-dabdd5ba-49c6-440f-bfd8-ca6ad17d876e None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:57.027586 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:57.029420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-dabdd5ba-49c6-440f-bfd8-ca6ad17d876e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:57.093794 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebb2fd-d0c7-4d3b-88b5-5d4628e042df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:57.097570 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebb2fd-d0c7-4d3b-88b5-5d4628e042df service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:57.116399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 968/1922] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:17:57.174721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 955/1923] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:57.186476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd65824-2518-4f9c-922b-f0b178fea300 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:57.187073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd65824-2518-4f9c-922b-f0b178fea300 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:57.187404 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd65824-2518-4f9c-922b-f0b178fea300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:57.208807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-1bd65824-2518-4f9c-922b-f0b178fea300 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:57.210283 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1bd65824-2518-4f9c-922b-f0b178fea300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:57.224556 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 969/1924] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:57.226774 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdd5ba-49c6-440f-bfd8-ca6ad17d876e None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:57.241146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdd5ba-49c6-440f-bfd8-ca6ad17d876e None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:57.247727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8b37c-a463-4dc3-a18a-c0b97d31ddac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:57.248706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8b37c-a463-4dc3-a18a-c0b97d31ddac admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:57.249138 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8b37c-a463-4dc3-a18a-c0b97d31ddac admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:57.249512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8b37c-a463-4dc3-a18a-c0b97d31ddac admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:57.250315 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e8b37c-a463-4dc3-a18a-c0b97d31ddac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:17:57.252901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e8b37c-a463-4dc3-a18a-c0b97d31ddac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:57.255442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e8b37c-a463-4dc3-a18a-c0b97d31ddac admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:57.255633 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd65824-2518-4f9c-922b-f0b178fea300 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:57.257399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1bd65824-2518-4f9c-922b-f0b178fea300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:57.269497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd65824-2518-4f9c-922b-f0b178fea300 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:57.277045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc25f2b-e72b-425b-af56-a5327529233c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:57.277218 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 956/1925] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:17:57.277302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc25f2b-e72b-425b-af56-a5327529233c service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:57.277302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc25f2b-e72b-425b-af56-a5327529233c service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:57.277302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc25f2b-e72b-425b-af56-a5327529233c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:57.277302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1bc25f2b-e72b-425b-af56-a5327529233c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:57.278567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8b37c-a463-4dc3-a18a-c0b97d31ddac admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:57.300579 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8b37c-a463-4dc3-a18a-c0b97d31ddac admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:57.309796 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabf100-4dea-42a3-bfb1-bc81fbb3f982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:57.310460 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabf100-4dea-42a3-bfb1-bc81fbb3f982 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:57.310765 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabf100-4dea-42a3-bfb1-bc81fbb3f982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:57.311181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabf100-4dea-42a3-bfb1-bc81fbb3f982 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:57.311880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabf100-4dea-42a3-bfb1-bc81fbb3f982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:57.313946 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabf100-4dea-42a3-bfb1-bc81fbb3f982 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:57.357364 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 957/1926] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:57 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:57.358517 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabf100-4dea-42a3-bfb1-bc81fbb3f982 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:57.364704 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc25f2b-e72b-425b-af56-a5327529233c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:57.367895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc25f2b-e72b-425b-af56-a5327529233c service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:57.384341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 970/1927] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:17:57.431650 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:57.447806 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabf100-4dea-42a3-bfb1-bc81fbb3f982 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:57.461991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da340fdb-67d7-4a23-bc99-56d4dced52eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:57.464318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da340fdb-67d7-4a23-bc99-56d4dced52eb admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:57.465522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da340fdb-67d7-4a23-bc99-56d4dced52eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:57.465899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da340fdb-67d7-4a23-bc99-56d4dced52eb admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:57.467575 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da340fdb-67d7-4a23-bc99-56d4dced52eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:57.469805 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da340fdb-67d7-4a23-bc99-56d4dced52eb admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:57.499569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 958/1928] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:57 2022] POST /identity/v3/users => generated 287 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:17:57.503762 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc25f2b-e72b-425b-af56-a5327529233c service cinder] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:57.518509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc25f2b-e72b-425b-af56-a5327529233c service cinder] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:57.529431 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fcad4-9616-4bfc-958e-045662977c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:57.530253 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fcad4-9616-4bfc-958e-045662977c2c admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:57.530663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fcad4-9616-4bfc-958e-045662977c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:57.531279 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fcad4-9616-4bfc-958e-045662977c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:57.532193 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fcad4-9616-4bfc-958e-045662977c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:57.534126 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fcad4-9616-4bfc-958e-045662977c2c admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:57.558732 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 971/1929] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:57.562993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da340fdb-67d7-4a23-bc99-56d4dced52eb admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:57.576465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da340fdb-67d7-4a23-bc99-56d4dced52eb admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:57.592733 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67c520-a29f-4f6c-af33-269d8a45cd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:57.593411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67c520-a29f-4f6c-af33-269d8a45cd18 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:57.593768 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67c520-a29f-4f6c-af33-269d8a45cd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:57.594244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67c520-a29f-4f6c-af33-269d8a45cd18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:57.594774 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67c520-a29f-4f6c-af33-269d8a45cd18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:57.596400 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67c520-a29f-4f6c-af33-269d8a45cd18 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:57.618880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 959/1930] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:57 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:17:57.619702 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fcad4-9616-4bfc-958e-045662977c2c admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:57.630925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fcad4-9616-4bfc-958e-045662977c2c admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:57.663600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fdfc7-9e07-45ae-b69e-6c98d1fdd853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:57.664313 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fdfc7-9e07-45ae-b69e-6c98d1fdd853 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:57.664654 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fdfc7-9e07-45ae-b69e-6c98d1fdd853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:57.664946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fdfc7-9e07-45ae-b69e-6c98d1fdd853 admin admin] PATH_INFO: `/v3/projects/1037d28132b3493bb9f26d1b1584cd66/users/5e2928a889d24b0283e7a203e8be0ef7/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:57.691242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432fdfc7-9e07-45ae-b69e-6c98d1fdd853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1037d28132b3493bb9f26d1b1584cd66, user_id=5e2928a889d24b0283e7a203e8be0ef7, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:57.694162 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432fdfc7-9e07-45ae-b69e-6c98d1fdd853 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:57.709584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 972/1931] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:17:57 2022] PUT /identity/v3/projects/1037d28132b3493bb9f26d1b1584cd66/users/5e2928a889d24b0283e7a203e8be0ef7/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:17:57.720753 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632dcd7-6157-4ff4-b4c8-799f190fd9a6 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:57.721045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632dcd7-6157-4ff4-b4c8-799f190fd9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:57.721328 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c632dcd7-6157-4ff4-b4c8-799f190fd9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:57.751669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-c632dcd7-6157-4ff4-b4c8-799f190fd9a6 None None] MFA Rules not processed for user `5e2928a889d24b0283e7a203e8be0ef7`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:57.752923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-c632dcd7-6157-4ff4-b4c8-799f190fd9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:57.826689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 960/1932] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:57.839557 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fdfc7-9e07-45ae-b69e-6c98d1fdd853 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:57.841492 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-432fdfc7-9e07-45ae-b69e-6c98d1fdd853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:57.909007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fdfc7-9e07-45ae-b69e-6c98d1fdd853 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:57.913421 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69edab9-0972-48e6-ba22-5d711bdf38f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:57.914598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69edab9-0972-48e6-ba22-5d711bdf38f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:57.914954 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69edab9-0972-48e6-ba22-5d711bdf38f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:57.915488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69edab9-0972-48e6-ba22-5d711bdf38f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:57.930454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69edab9-0972-48e6-ba22-5d711bdf38f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:57.932311 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69edab9-0972-48e6-ba22-5d711bdf38f9 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:57.942584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 973/1933] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:57.966093 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632dcd7-6157-4ff4-b4c8-799f190fd9a6 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:58.055390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c632dcd7-6157-4ff4-b4c8-799f190fd9a6 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:58.060745 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46dcce-f293-4167-98a8-c0724978db56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:58.061633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46dcce-f293-4167-98a8-c0724978db56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:58.061970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46dcce-f293-4167-98a8-c0724978db56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:58.062313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad46dcce-f293-4167-98a8-c0724978db56 admin admin] PATH_INFO: `/v3/users/5e2928a889d24b0283e7a203e8be0ef7` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:58.063800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46dcce-f293-4167-98a8-c0724978db56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e2928a889d24b0283e7a203e8be0ef7)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:58.065912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad46dcce-f293-4167-98a8-c0724978db56 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:58.126336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ad46dcce-f293-4167-98a8-c0724978db56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e2928a889d24b0283e7a203e8be0ef7'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:58.127072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ad46dcce-f293-4167-98a8-c0724978db56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e2928a889d24b0283e7a203e8be0ef7'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:58.127825 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ad46dcce-f293-4167-98a8-c0724978db56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e2928a889d24b0283e7a203e8be0ef7'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:58.141436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:58.143188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ad46dcce-f293-4167-98a8-c0724978db56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e2928a889d24b0283e7a203e8be0ef7'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:58.153177 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ad46dcce-f293-4167-98a8-c0724978db56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e2928a889d24b0283e7a203e8be0ef7'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:58.171160 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 961/1934] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:57 2022] DELETE /identity/v3/users/5e2928a889d24b0283e7a203e8be0ef7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:58.175443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d5187-b5d9-4b5c-af5c-fbc18f73b6c4 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:58.175873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d5187-b5d9-4b5c-af5c-fbc18f73b6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:58.176219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d5187-b5d9-4b5c-af5c-fbc18f73b6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:58.203422 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-554d5187-b5d9-4b5c-af5c-fbc18f73b6c4 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:58.204862 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-554d5187-b5d9-4b5c-af5c-fbc18f73b6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:58.222567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:58.284143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 974/1935] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:58.294027 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46dcce-f293-4167-98a8-c0724978db56 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:58.295523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ad46dcce-f293-4167-98a8-c0724978db56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:58.395689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad46dcce-f293-4167-98a8-c0724978db56 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:58.400919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8b60d-ae69-4b75-8a54-60d99cdcdeb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:58.403135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8b60d-ae69-4b75-8a54-60d99cdcdeb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:58.403762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8b60d-ae69-4b75-8a54-60d99cdcdeb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:58.404219 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8b60d-ae69-4b75-8a54-60d99cdcdeb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:58.421739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a8b60d-ae69-4b75-8a54-60d99cdcdeb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:58.424803 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a8b60d-ae69-4b75-8a54-60d99cdcdeb0 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:58.440238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 962/1936] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:17:58.500338 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d5187-b5d9-4b5c-af5c-fbc18f73b6c4 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:58.502144 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-554d5187-b5d9-4b5c-af5c-fbc18f73b6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:58.520145 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d5187-b5d9-4b5c-af5c-fbc18f73b6c4 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:58.526335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1b399-1d94-467d-8b35-843472abe4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:58.527225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1b399-1d94-467d-8b35-843472abe4f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:58.527660 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1b399-1d94-467d-8b35-843472abe4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:58.528087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1b399-1d94-467d-8b35-843472abe4f3 admin admin] PATH_INFO: `/v3/users/9eb9b935f4324c26b7f7ad7ff256630c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:58.530464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1b399-1d94-467d-8b35-843472abe4f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eb9b935f4324c26b7f7ad7ff256630c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:58.533539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1b399-1d94-467d-8b35-843472abe4f3 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:58.631961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-9de1b399-1d94-467d-8b35-843472abe4f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eb9b935f4324c26b7f7ad7ff256630c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:58.669401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-9de1b399-1d94-467d-8b35-843472abe4f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eb9b935f4324c26b7f7ad7ff256630c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:58.671461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-9de1b399-1d94-467d-8b35-843472abe4f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eb9b935f4324c26b7f7ad7ff256630c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:58.718176 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-9de1b399-1d94-467d-8b35-843472abe4f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eb9b935f4324c26b7f7ad7ff256630c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:58.719019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-9de1b399-1d94-467d-8b35-843472abe4f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eb9b935f4324c26b7f7ad7ff256630c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:58.743998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 975/1937] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:58 2022] DELETE /identity/v3/users/9eb9b935f4324c26b7f7ad7ff256630c => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:58.750167 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71541499-ac73-444d-89fe-2e2eafb2a942 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:58.750608 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71541499-ac73-444d-89fe-2e2eafb2a942 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:58.751031 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71541499-ac73-444d-89fe-2e2eafb2a942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:58.798656 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-71541499-ac73-444d-89fe-2e2eafb2a942 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:58.805327 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-71541499-ac73-444d-89fe-2e2eafb2a942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:58.891501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 963/1938] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:58.916094 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1b399-1d94-467d-8b35-843472abe4f3 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:58.917655 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9de1b399-1d94-467d-8b35-843472abe4f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:59.061936 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1b399-1d94-467d-8b35-843472abe4f3 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:59.066301 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d3889-8de8-4457-90c8-e1c9e42e5cf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:59.067124 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d3889-8de8-4457-90c8-e1c9e42e5cf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:59.067559 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d3889-8de8-4457-90c8-e1c9e42e5cf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:59.068082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d3889-8de8-4457-90c8-e1c9e42e5cf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:59.080596 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d3889-8de8-4457-90c8-e1c9e42e5cf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:59.083017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d3889-8de8-4457-90c8-e1c9e42e5cf4 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:59.098425 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 976/1939] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:17:59.201469 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71541499-ac73-444d-89fe-2e2eafb2a942 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:59.202965 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-71541499-ac73-444d-89fe-2e2eafb2a942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:59.226230 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71541499-ac73-444d-89fe-2e2eafb2a942 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:59.234368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096f830-74d6-4d90-a4f8-c56bc4bb8ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:59.235026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096f830-74d6-4d90-a4f8-c56bc4bb8ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:59.235445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096f830-74d6-4d90-a4f8-c56bc4bb8ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:59.235730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096f830-74d6-4d90-a4f8-c56bc4bb8ef9 admin admin] PATH_INFO: `/v3/projects/1037d28132b3493bb9f26d1b1584cd66` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:59.237414 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1096f830-74d6-4d90-a4f8-c56bc4bb8ef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1037d28132b3493bb9f26d1b1584cd66)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:59.239341 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1096f830-74d6-4d90-a4f8-c56bc4bb8ef9 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:59.419603 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1096f830-74d6-4d90-a4f8-c56bc4bb8ef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1037d28132b3493bb9f26d1b1584cd66', 'request_id': 'req-1096f830-74d6-4d90-a4f8-c56bc4bb8ef9'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:59.439173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1096f830-74d6-4d90-a4f8-c56bc4bb8ef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1037d28132b3493bb9f26d1b1584cd66', 'request_id': 'req-1096f830-74d6-4d90-a4f8-c56bc4bb8ef9'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:59.450447 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1096f830-74d6-4d90-a4f8-c56bc4bb8ef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1037d28132b3493bb9f26d1b1584cd66', 'request_id': 'req-1096f830-74d6-4d90-a4f8-c56bc4bb8ef9'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:59.470424 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1096f830-74d6-4d90-a4f8-c56bc4bb8ef9 admin admin] The token cache is being invalidate because project 1037d28132b3493bb9f26d1b1584cd66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:59.470726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1096f830-74d6-4d90-a4f8-c56bc4bb8ef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:59.471551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1096f830-74d6-4d90-a4f8-c56bc4bb8ef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:59.477738 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 964/1940] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:59 2022] DELETE /identity/v3/projects/1037d28132b3493bb9f26d1b1584cd66 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:59.478921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b563194-4e48-40f7-90df-c193c1924d28 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:59.479205 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b563194-4e48-40f7-90df-c193c1924d28 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:59.479474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b563194-4e48-40f7-90df-c193c1924d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:59.512320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-3b563194-4e48-40f7-90df-c193c1924d28 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:59.513957 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-3b563194-4e48-40f7-90df-c193c1924d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:59.566853 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:59.614936 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 977/1941] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:17:59.622939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c3d67-ac82-49bd-b1e7-e7ba9c5fe6b3 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:59.622939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c3d67-ac82-49bd-b1e7-e7ba9c5fe6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:59.622939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c3d67-ac82-49bd-b1e7-e7ba9c5fe6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:59.641523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:17:59.650720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-302c3d67-ac82-49bd-b1e7-e7ba9c5fe6b3 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:17:59.655785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-302c3d67-ac82-49bd-b1e7-e7ba9c5fe6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:59.681685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 965/1942] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:17:59.686614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b563194-4e48-40f7-90df-c193c1924d28 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:59.695121 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c3d67-ac82-49bd-b1e7-e7ba9c5fe6b3 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:59.696738 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-302c3d67-ac82-49bd-b1e7-e7ba9c5fe6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:59.706231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b563194-4e48-40f7-90df-c193c1924d28 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:59.716923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccc62e-eda4-471e-957c-8eb30e24ace7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:59.719272 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccc62e-eda4-471e-957c-8eb30e24ace7 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:59.719272 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccc62e-eda4-471e-957c-8eb30e24ace7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:59.719272 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccc62e-eda4-471e-957c-8eb30e24ace7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:59.719692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccc62e-eda4-471e-957c-8eb30e24ace7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:17:59.720005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccc62e-eda4-471e-957c-8eb30e24ace7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:59.721928 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccc62e-eda4-471e-957c-8eb30e24ace7 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:59.751902 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 978/1943] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:17:59.754647 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c3d67-ac82-49bd-b1e7-e7ba9c5fe6b3 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:59.756053 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccc62e-eda4-471e-957c-8eb30e24ace7 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:59.767457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38076f7-c671-4f11-b3ab-d34dd16778a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:59.768050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38076f7-c671-4f11-b3ab-d34dd16778a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:59.768332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38076f7-c671-4f11-b3ab-d34dd16778a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:59.768690 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38076f7-c671-4f11-b3ab-d34dd16778a6 admin admin] PATH_INFO: `/v3/projects/37ae7a75818544b2809031243e85749a` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:59.774118 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccc62e-eda4-471e-957c-8eb30e24ace7 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:59.775403 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38076f7-c671-4f11-b3ab-d34dd16778a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37ae7a75818544b2809031243e85749a)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:59.777616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38076f7-c671-4f11-b3ab-d34dd16778a6 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:59.787467 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87f55a-7e0f-48b6-894d-feb04523d2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:59.788110 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87f55a-7e0f-48b6-894d-feb04523d2cc admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:59.791419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87f55a-7e0f-48b6-894d-feb04523d2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:59.791738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87f55a-7e0f-48b6-894d-feb04523d2cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:59.792443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b87f55a-7e0f-48b6-894d-feb04523d2cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:59.794541 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b87f55a-7e0f-48b6-894d-feb04523d2cc admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:59.841161 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 979/1944] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:17:59 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:17:59.846151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87f55a-7e0f-48b6-894d-feb04523d2cc admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:59.854252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d38076f7-c671-4f11-b3ab-d34dd16778a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37ae7a75818544b2809031243e85749a', 'request_id': 'req-d38076f7-c671-4f11-b3ab-d34dd16778a6'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:59.864190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d38076f7-c671-4f11-b3ab-d34dd16778a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37ae7a75818544b2809031243e85749a', 'request_id': 'req-d38076f7-c671-4f11-b3ab-d34dd16778a6'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:59.872596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d38076f7-c671-4f11-b3ab-d34dd16778a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37ae7a75818544b2809031243e85749a', 'request_id': 'req-d38076f7-c671-4f11-b3ab-d34dd16778a6'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:59.886488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d38076f7-c671-4f11-b3ab-d34dd16778a6 admin admin] The token cache is being invalidate because project 37ae7a75818544b2809031243e85749a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:17:59.886783 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d38076f7-c671-4f11-b3ab-d34dd16778a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:59.887665 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d38076f7-c671-4f11-b3ab-d34dd16778a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:17:59.890098 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 966/1945] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:17:59 2022] DELETE /identity/v3/projects/37ae7a75818544b2809031243e85749a => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:17:59.894042 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38076f7-c671-4f11-b3ab-d34dd16778a6 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:59.895680 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d38076f7-c671-4f11-b3ab-d34dd16778a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:59.924130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87f55a-7e0f-48b6-894d-feb04523d2cc admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:59.928940 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80277a77-7ae1-4721-b774-953fbf968db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:59.929600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80277a77-7ae1-4721-b774-953fbf968db9 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:59.929880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80277a77-7ae1-4721-b774-953fbf968db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:59.930153 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80277a77-7ae1-4721-b774-953fbf968db9 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:59.930816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80277a77-7ae1-4721-b774-953fbf968db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:59.932800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80277a77-7ae1-4721-b774-953fbf968db9 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:17:59.965681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 980/1946] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:17:59 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:17:59.967071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80277a77-7ae1-4721-b774-953fbf968db9 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:17:59.968896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-80277a77-7ae1-4721-b774-953fbf968db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:17:59.976965 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38076f7-c671-4f11-b3ab-d34dd16778a6 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:17:59.983730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a874df-1a03-411f-8508-3d14256b15f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:17:59.984395 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a874df-1a03-411f-8508-3d14256b15f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:17:59.984722 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a874df-1a03-411f-8508-3d14256b15f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:17:59.984995 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a874df-1a03-411f-8508-3d14256b15f4 admin admin] PATH_INFO: `/v3/users/d9885632a3ac46779e475ddbffe3ef17` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:17:59.986223 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a874df-1a03-411f-8508-3d14256b15f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9885632a3ac46779e475ddbffe3ef17)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:17:59.988664 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a874df-1a03-411f-8508-3d14256b15f4 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:00.004921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80277a77-7ae1-4721-b774-953fbf968db9 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:00.014546 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25692b5-06e0-4bb0-8120-a96d6f79faac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:00.015769 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25692b5-06e0-4bb0-8120-a96d6f79faac admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:00.016355 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25692b5-06e0-4bb0-8120-a96d6f79faac admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:00.017028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25692b5-06e0-4bb0-8120-a96d6f79faac admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:00.018054 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25692b5-06e0-4bb0-8120-a96d6f79faac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:00.021813 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25692b5-06e0-4bb0-8120-a96d6f79faac admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:00.041226 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 981/1947] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:18:00.042554 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25692b5-06e0-4bb0-8120-a96d6f79faac admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:00.054216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25692b5-06e0-4bb0-8120-a96d6f79faac admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:00.057330 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-28a874df-1a03-411f-8508-3d14256b15f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9885632a3ac46779e475ddbffe3ef17'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:00.058087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-28a874df-1a03-411f-8508-3d14256b15f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9885632a3ac46779e475ddbffe3ef17'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:00.058769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-28a874df-1a03-411f-8508-3d14256b15f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9885632a3ac46779e475ddbffe3ef17'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:00.060502 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e83f74-3a67-4871-83a7-b3a40c131461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:00.061478 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e83f74-3a67-4871-83a7-b3a40c131461 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:00.061922 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e83f74-3a67-4871-83a7-b3a40c131461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:00.062202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e83f74-3a67-4871-83a7-b3a40c131461 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:00.062922 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e83f74-3a67-4871-83a7-b3a40c131461 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:00.065010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e83f74-3a67-4871-83a7-b3a40c131461 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:00.077217 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-28a874df-1a03-411f-8508-3d14256b15f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9885632a3ac46779e475ddbffe3ef17'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:00.078550 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 982/1948] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:00.083317 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e83f74-3a67-4871-83a7-b3a40c131461 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:00.085284 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-63e83f74-3a67-4871-83a7-b3a40c131461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:00.086802 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-28a874df-1a03-411f-8508-3d14256b15f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9885632a3ac46779e475ddbffe3ef17'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:00.102112 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e83f74-3a67-4871-83a7-b3a40c131461 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:00.108922 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 967/1949] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:17:59 2022] DELETE /identity/v3/users/d9885632a3ac46779e475ddbffe3ef17 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:18:00.113015 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb97a1-819d-41e3-b529-1d1565be418d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:00.114590 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb97a1-819d-41e3-b529-1d1565be418d admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:00.114976 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb97a1-819d-41e3-b529-1d1565be418d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:00.115355 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb97a1-819d-41e3-b529-1d1565be418d admin admin] PATH_INFO: `/v3/projects/6f2c985de2e84620a80a708ce6c8240e/users/9a927cd4c01d4fc3ac0752c2b14b1994/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:00.138303 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb97a1-819d-41e3-b529-1d1565be418d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f2c985de2e84620a80a708ce6c8240e, user_id=9a927cd4c01d4fc3ac0752c2b14b1994, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:00.143678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb97a1-819d-41e3-b529-1d1565be418d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:00.166809 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 983/1950] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:00 2022] PUT /identity/v3/projects/6f2c985de2e84620a80a708ce6c8240e/users/9a927cd4c01d4fc3ac0752c2b14b1994/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:00.179201 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85959d88-0308-4adc-9a76-5107480362e6 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:00.181123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85959d88-0308-4adc-9a76-5107480362e6 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:00.181123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85959d88-0308-4adc-9a76-5107480362e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:00.194396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:00.249559 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-85959d88-0308-4adc-9a76-5107480362e6 None None] MFA Rules not processed for user `9a927cd4c01d4fc3ac0752c2b14b1994`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:00.253457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-85959d88-0308-4adc-9a76-5107480362e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:00.357908 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 968/1951] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:00.369871 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb97a1-819d-41e3-b529-1d1565be418d admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:00.372000 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-91eb97a1-819d-41e3-b529-1d1565be418d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:00.426523 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:00.442947 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb97a1-819d-41e3-b529-1d1565be418d admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:00.449529 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f943c53-3cfb-4f53-83de-61829c564c25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:00.450300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f943c53-3cfb-4f53-83de-61829c564c25 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:00.450679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f943c53-3cfb-4f53-83de-61829c564c25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:00.451140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f943c53-3cfb-4f53-83de-61829c564c25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:00.467190 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f943c53-3cfb-4f53-83de-61829c564c25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:00.473894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f943c53-3cfb-4f53-83de-61829c564c25 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:00.494669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 984/1952] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:00.670728 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85959d88-0308-4adc-9a76-5107480362e6 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:00.673265 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f943c53-3cfb-4f53-83de-61829c564c25 service cinder] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:00.674926 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1f943c53-3cfb-4f53-83de-61829c564c25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:00.675567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-85959d88-0308-4adc-9a76-5107480362e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:00.817545 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85959d88-0308-4adc-9a76-5107480362e6 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:00.827161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4d5a7-6229-4306-8795-f86ccb64bde1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:00.827835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4d5a7-6229-4306-8795-f86ccb64bde1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:00.828118 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4d5a7-6229-4306-8795-f86ccb64bde1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:00.828396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc4d5a7-6229-4306-8795-f86ccb64bde1 admin admin] PATH_INFO: `/v3/projects/3ee3e0b03cfc44dc8b321951feef3a61` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:00.829663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4d5a7-6229-4306-8795-f86ccb64bde1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ee3e0b03cfc44dc8b321951feef3a61)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:00.831764 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc4d5a7-6229-4306-8795-f86ccb64bde1 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:00.837211 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f943c53-3cfb-4f53-83de-61829c564c25 service cinder] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:00.847040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af98625-b1e0-4444-b665-33a58591257e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:00.847801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af98625-b1e0-4444-b665-33a58591257e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:00.848918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af98625-b1e0-4444-b665-33a58591257e admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:00.849865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af98625-b1e0-4444-b665-33a58591257e admin admin] PATH_INFO: `/v3/users/21a0f892e60c463586c7d919eaaa8be0` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:00.856100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af98625-b1e0-4444-b665-33a58591257e admin admin] RBAC: Authorizing `identity:delete_user(user_id=21a0f892e60c463586c7d919eaaa8be0)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:00.861365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af98625-b1e0-4444-b665-33a58591257e admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:00.934326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4dc4d5a7-6229-4306-8795-f86ccb64bde1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ee3e0b03cfc44dc8b321951feef3a61', 'request_id': 'req-4dc4d5a7-6229-4306-8795-f86ccb64bde1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:00.969781 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4dc4d5a7-6229-4306-8795-f86ccb64bde1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ee3e0b03cfc44dc8b321951feef3a61', 'request_id': 'req-4dc4d5a7-6229-4306-8795-f86ccb64bde1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:00.979616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4dc4d5a7-6229-4306-8795-f86ccb64bde1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ee3e0b03cfc44dc8b321951feef3a61', 'request_id': 'req-4dc4d5a7-6229-4306-8795-f86ccb64bde1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:00.995325 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2af98625-b1e0-4444-b665-33a58591257e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21a0f892e60c463586c7d919eaaa8be0'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:00.999165 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4dc4d5a7-6229-4306-8795-f86ccb64bde1 admin admin] The token cache is being invalidate because project 3ee3e0b03cfc44dc8b321951feef3a61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:00.999165 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4dc4d5a7-6229-4306-8795-f86ccb64bde1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.000831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4dc4d5a7-6229-4306-8795-f86ccb64bde1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.009742 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 969/1953] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:00 2022] DELETE /identity/v3/projects/3ee3e0b03cfc44dc8b321951feef3a61 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:01.010308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4d5a7-6229-4306-8795-f86ccb64bde1 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:01.011718 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-4dc4d5a7-6229-4306-8795-f86ccb64bde1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:01.017673 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2af98625-b1e0-4444-b665-33a58591257e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21a0f892e60c463586c7d919eaaa8be0'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.018890 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2af98625-b1e0-4444-b665-33a58591257e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21a0f892e60c463586c7d919eaaa8be0'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.034531 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2af98625-b1e0-4444-b665-33a58591257e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21a0f892e60c463586c7d919eaaa8be0'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.035263 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2af98625-b1e0-4444-b665-33a58591257e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21a0f892e60c463586c7d919eaaa8be0'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.054983 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 985/1954] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:18:00 2022] DELETE /identity/v3/users/21a0f892e60c463586c7d919eaaa8be0 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:01.058831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:01.061394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83893000-477e-4386-a28e-9d9b8fce3114 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:01.061739 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83893000-477e-4386-a28e-9d9b8fce3114 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:01.062391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83893000-477e-4386-a28e-9d9b8fce3114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:01.088058 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-83893000-477e-4386-a28e-9d9b8fce3114 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:01.089835 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-83893000-477e-4386-a28e-9d9b8fce3114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:01.126348 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc4d5a7-6229-4306-8795-f86ccb64bde1 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:01.192779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 986/1955] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:01.200631 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232c8bd-d35d-411f-82c1-85b73d38e365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:01.201560 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232c8bd-d35d-411f-82c1-85b73d38e365 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:01.201962 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232c8bd-d35d-411f-82c1-85b73d38e365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:01.202339 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232c8bd-d35d-411f-82c1-85b73d38e365 admin admin] PATH_INFO: `/v3/users/57738349f9f34c12b35d397c55ec9130` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:01.204101 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232c8bd-d35d-411f-82c1-85b73d38e365 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57738349f9f34c12b35d397c55ec9130)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:01.205089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83893000-477e-4386-a28e-9d9b8fce3114 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:01.206676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-83893000-477e-4386-a28e-9d9b8fce3114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:01.206810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232c8bd-d35d-411f-82c1-85b73d38e365 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:01.224342 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:01.294937 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83893000-477e-4386-a28e-9d9b8fce3114 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:01.300981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804ae4c-bf67-4c22-ba24-6874c4940dc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:01.301725 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804ae4c-bf67-4c22-ba24-6874c4940dc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:01.302011 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804ae4c-bf67-4c22-ba24-6874c4940dc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:01.302287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e804ae4c-bf67-4c22-ba24-6874c4940dc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:01.305847 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-7232c8bd-d35d-411f-82c1-85b73d38e365 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57738349f9f34c12b35d397c55ec9130'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.306986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-7232c8bd-d35d-411f-82c1-85b73d38e365 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57738349f9f34c12b35d397c55ec9130'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.307813 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-7232c8bd-d35d-411f-82c1-85b73d38e365 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57738349f9f34c12b35d397c55ec9130'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.314830 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804ae4c-bf67-4c22-ba24-6874c4940dc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:01.316331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e804ae4c-bf67-4c22-ba24-6874c4940dc4 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:01.324442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-7232c8bd-d35d-411f-82c1-85b73d38e365 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57738349f9f34c12b35d397c55ec9130'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.329820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 987/1956] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:18:01.332993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-7232c8bd-d35d-411f-82c1-85b73d38e365 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57738349f9f34c12b35d397c55ec9130'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.352433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 970/1957] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:18:01 2022] DELETE /identity/v3/users/57738349f9f34c12b35d397c55ec9130 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:01.635438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804ae4c-bf67-4c22-ba24-6874c4940dc4 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:01.637017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e804ae4c-bf67-4c22-ba24-6874c4940dc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:01.665421 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e804ae4c-bf67-4c22-ba24-6874c4940dc4 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:01.686949 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232c8bd-d35d-411f-82c1-85b73d38e365 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:01.688502 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7232c8bd-d35d-411f-82c1-85b73d38e365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:01.735763 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa2330-91cd-410e-91ee-912e5f9b86c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:01.736439 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa2330-91cd-410e-91ee-912e5f9b86c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:01.736766 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa2330-91cd-410e-91ee-912e5f9b86c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:01.737045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa2330-91cd-410e-91ee-912e5f9b86c8 admin admin] PATH_INFO: `/v3/projects/5bc9909ab42b4e578faa517387780320` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:01.738210 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa2330-91cd-410e-91ee-912e5f9b86c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bc9909ab42b4e578faa517387780320)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:01.740102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa2330-91cd-410e-91ee-912e5f9b86c8 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:01.755278 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:01.774271 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232c8bd-d35d-411f-82c1-85b73d38e365 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:01.780374 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e30b8-f754-4b5a-88af-13b848d45e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:01.783710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e30b8-f754-4b5a-88af-13b848d45e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:01.784094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e30b8-f754-4b5a-88af-13b848d45e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:01.784570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e30b8-f754-4b5a-88af-13b848d45e7c admin admin] PATH_INFO: `/v3/projects/c2644fb449d6470291261892b7a46a6c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:01.786217 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89e30b8-f754-4b5a-88af-13b848d45e7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2644fb449d6470291261892b7a46a6c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:01.788942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89e30b8-f754-4b5a-88af-13b848d45e7c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:01.814358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-7afa2330-91cd-410e-91ee-912e5f9b86c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bc9909ab42b4e578faa517387780320', 'request_id': 'req-7afa2330-91cd-410e-91ee-912e5f9b86c8'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.825481 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-7afa2330-91cd-410e-91ee-912e5f9b86c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bc9909ab42b4e578faa517387780320', 'request_id': 'req-7afa2330-91cd-410e-91ee-912e5f9b86c8'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.833089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-7afa2330-91cd-410e-91ee-912e5f9b86c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bc9909ab42b4e578faa517387780320', 'request_id': 'req-7afa2330-91cd-410e-91ee-912e5f9b86c8'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.846062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-7afa2330-91cd-410e-91ee-912e5f9b86c8 admin admin] The token cache is being invalidate because project 5bc9909ab42b4e578faa517387780320 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:01.846439 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-7afa2330-91cd-410e-91ee-912e5f9b86c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.846545 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-f89e30b8-f754-4b5a-88af-13b848d45e7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2644fb449d6470291261892b7a46a6c', 'request_id': 'req-f89e30b8-f754-4b5a-88af-13b848d45e7c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.847224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-7afa2330-91cd-410e-91ee-912e5f9b86c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.848867 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 988/1958] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:01 2022] DELETE /identity/v3/projects/5bc9909ab42b4e578faa517387780320 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:01.853666 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa2330-91cd-410e-91ee-912e5f9b86c8 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:01.855316 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7afa2330-91cd-410e-91ee-912e5f9b86c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:01.855904 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-f89e30b8-f754-4b5a-88af-13b848d45e7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2644fb449d6470291261892b7a46a6c', 'request_id': 'req-f89e30b8-f754-4b5a-88af-13b848d45e7c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.863262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-f89e30b8-f754-4b5a-88af-13b848d45e7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2644fb449d6470291261892b7a46a6c', 'request_id': 'req-f89e30b8-f754-4b5a-88af-13b848d45e7c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.875296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-f89e30b8-f754-4b5a-88af-13b848d45e7c admin admin] The token cache is being invalidate because project c2644fb449d6470291261892b7a46a6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:01.875586 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-f89e30b8-f754-4b5a-88af-13b848d45e7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.876248 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-f89e30b8-f754-4b5a-88af-13b848d45e7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:01.881350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 971/1959] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:01 2022] DELETE /identity/v3/projects/c2644fb449d6470291261892b7a46a6c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:18:01.882818 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff100dd-f94f-4418-8e7e-6b2be99ea2e6 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:01.884710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff100dd-f94f-4418-8e7e-6b2be99ea2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:01.884890 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff100dd-f94f-4418-8e7e-6b2be99ea2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:01.905073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-3ff100dd-f94f-4418-8e7e-6b2be99ea2e6 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:01.906390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-3ff100dd-f94f-4418-8e7e-6b2be99ea2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:01.910108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:01.938664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa2330-91cd-410e-91ee-912e5f9b86c8 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:02.004656 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 972/1960] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:18:02.014617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1070e29-c0ca-4eb3-8ebd-e9bc53eae492 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.015045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1070e29-c0ca-4eb3-8ebd-e9bc53eae492 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.015425 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1070e29-c0ca-4eb3-8ebd-e9bc53eae492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.046652 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-c1070e29-c0ca-4eb3-8ebd-e9bc53eae492 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:02.049787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-c1070e29-c0ca-4eb3-8ebd-e9bc53eae492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:02.068173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab9d04-0931-4fa4-aa31-1d1bb6d402c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:02.068173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab9d04-0931-4fa4-aa31-1d1bb6d402c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.068173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab9d04-0931-4fa4-aa31-1d1bb6d402c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.068173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ab9d04-0931-4fa4-aa31-1d1bb6d402c0 admin admin] PATH_INFO: `/v3/users/30e2a481134a494b8f2a8a5dc5433253` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.074074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab9d04-0931-4fa4-aa31-1d1bb6d402c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30e2a481134a494b8f2a8a5dc5433253)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:02.080836 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 973/1961] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:02.081018 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ab9d04-0931-4fa4-aa31-1d1bb6d402c0 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:02.082039 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1070e29-c0ca-4eb3-8ebd-e9bc53eae492 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:02.097747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1070e29-c0ca-4eb3-8ebd-e9bc53eae492 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:02.107419 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe1793-87f0-4c6f-8050-16681ef48deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:02.108186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe1793-87f0-4c6f-8050-16681ef48deb admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.108552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe1793-87f0-4c6f-8050-16681ef48deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.108945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fe1793-87f0-4c6f-8050-16681ef48deb admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.110181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe1793-87f0-4c6f-8050-16681ef48deb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:18:02.110590 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe1793-87f0-4c6f-8050-16681ef48deb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:02.114005 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fe1793-87f0-4c6f-8050-16681ef48deb admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:02.129755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 974/1962] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:18:02.133926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe1793-87f0-4c6f-8050-16681ef48deb admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:02.144692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fe1793-87f0-4c6f-8050-16681ef48deb admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:02.149911 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-33ab9d04-0931-4fa4-aa31-1d1bb6d402c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30e2a481134a494b8f2a8a5dc5433253'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:02.151045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b6d7e-05cc-4912-9506-cf9942505c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:02.151554 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b6d7e-05cc-4912-9506-cf9942505c41 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.152040 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b6d7e-05cc-4912-9506-cf9942505c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.152526 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b6d7e-05cc-4912-9506-cf9942505c41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.153653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b6d7e-05cc-4912-9506-cf9942505c41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:02.156355 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b6d7e-05cc-4912-9506-cf9942505c41 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:02.158997 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-33ab9d04-0931-4fa4-aa31-1d1bb6d402c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30e2a481134a494b8f2a8a5dc5433253'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:02.159741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-33ab9d04-0931-4fa4-aa31-1d1bb6d402c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30e2a481134a494b8f2a8a5dc5433253'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:02.178037 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-33ab9d04-0931-4fa4-aa31-1d1bb6d402c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30e2a481134a494b8f2a8a5dc5433253'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:02.178924 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-33ab9d04-0931-4fa4-aa31-1d1bb6d402c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30e2a481134a494b8f2a8a5dc5433253'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:02.192730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 975/1963] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:18:02 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:02.197803 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b6d7e-05cc-4912-9506-cf9942505c41 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:02.199990 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-013b6d7e-05cc-4912-9506-cf9942505c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:02.201141 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 989/1964] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:18:01 2022] DELETE /identity/v3/users/30e2a481134a494b8f2a8a5dc5433253 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:02.286077 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:02.316870 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b6d7e-05cc-4912-9506-cf9942505c41 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:02.421404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9124d3-578e-495e-bf7c-aea45ed5916f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:02.422312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9124d3-578e-495e-bf7c-aea45ed5916f admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.422893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9124d3-578e-495e-bf7c-aea45ed5916f admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.423094 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9124d3-578e-495e-bf7c-aea45ed5916f admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.423715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9124d3-578e-495e-bf7c-aea45ed5916f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:02.425676 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9124d3-578e-495e-bf7c-aea45ed5916f admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:02.448909 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 976/1965] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:18:02 2022] POST /identity/v3/users => generated 286 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:18:02.452921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab9d04-0931-4fa4-aa31-1d1bb6d402c0 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:02.464259 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ab9d04-0931-4fa4-aa31-1d1bb6d402c0 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:02.468838 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6174fc-8476-43c5-902f-f747c4d15882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:02.469480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6174fc-8476-43c5-902f-f747c4d15882 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.469740 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6174fc-8476-43c5-902f-f747c4d15882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.470017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6174fc-8476-43c5-902f-f747c4d15882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.470752 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6174fc-8476-43c5-902f-f747c4d15882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:02.472498 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6174fc-8476-43c5-902f-f747c4d15882 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:02.483126 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 990/1966] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:02.487927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9124d3-578e-495e-bf7c-aea45ed5916f admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:02.498391 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9124d3-578e-495e-bf7c-aea45ed5916f admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:02.503379 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71725857-3a07-4385-954c-e11d8c31e4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:02.504030 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71725857-3a07-4385-954c-e11d8c31e4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.504309 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71725857-3a07-4385-954c-e11d8c31e4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.504580 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71725857-3a07-4385-954c-e11d8c31e4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.505308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71725857-3a07-4385-954c-e11d8c31e4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:02.507014 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71725857-3a07-4385-954c-e11d8c31e4e0 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:02.518367 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 977/1967] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:18:02.522968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6174fc-8476-43c5-902f-f747c4d15882 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:02.526664 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71725857-3a07-4385-954c-e11d8c31e4e0 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:02.528675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-71725857-3a07-4385-954c-e11d8c31e4e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:02.534752 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6174fc-8476-43c5-902f-f747c4d15882 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:02.542370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45db0b-a757-483b-a57e-4d145307bdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:02.543665 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45db0b-a757-483b-a57e-4d145307bdfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.544015 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71725857-3a07-4385-954c-e11d8c31e4e0 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:02.544491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45db0b-a757-483b-a57e-4d145307bdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.545020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45db0b-a757-483b-a57e-4d145307bdfc admin admin] PATH_INFO: `/v3/projects/d7f54ee4ba204ceb982f8e4ad5dee102/users/11bf0abff172401bb62653e975c91f45/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.559108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0514cb19-b6bc-4e1c-8c3a-5003e1f26297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:02.559908 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514cb19-b6bc-4e1c-8c3a-5003e1f26297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.560108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514cb19-b6bc-4e1c-8c3a-5003e1f26297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.560402 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0514cb19-b6bc-4e1c-8c3a-5003e1f26297 admin admin] PATH_INFO: `/v3/projects/ff9c9354e0164e55ae92308199240cb7` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.561588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514cb19-b6bc-4e1c-8c3a-5003e1f26297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff9c9354e0164e55ae92308199240cb7)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:02.563654 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0514cb19-b6bc-4e1c-8c3a-5003e1f26297 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:02.565304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45db0b-a757-483b-a57e-4d145307bdfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7f54ee4ba204ceb982f8e4ad5dee102, user_id=11bf0abff172401bb62653e975c91f45, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:02.568112 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45db0b-a757-483b-a57e-4d145307bdfc admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:02.584110 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 991/1968] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:02 2022] PUT /identity/v3/projects/d7f54ee4ba204ceb982f8e4ad5dee102/users/11bf0abff172401bb62653e975c91f45/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:02.589009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3d4ce-1804-4cca-95ce-0da346353dbf None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.589293 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3d4ce-1804-4cca-95ce-0da346353dbf None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.589565 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa3d4ce-1804-4cca-95ce-0da346353dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.608182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-1fa3d4ce-1804-4cca-95ce-0da346353dbf None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:02.609466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1fa3d4ce-1804-4cca-95ce-0da346353dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:02.615769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0514cb19-b6bc-4e1c-8c3a-5003e1f26297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff9c9354e0164e55ae92308199240cb7', 'request_id': 'req-0514cb19-b6bc-4e1c-8c3a-5003e1f26297'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:02.625585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0514cb19-b6bc-4e1c-8c3a-5003e1f26297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff9c9354e0164e55ae92308199240cb7', 'request_id': 'req-0514cb19-b6bc-4e1c-8c3a-5003e1f26297'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:02.632320 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0514cb19-b6bc-4e1c-8c3a-5003e1f26297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff9c9354e0164e55ae92308199240cb7', 'request_id': 'req-0514cb19-b6bc-4e1c-8c3a-5003e1f26297'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:02.647258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:02.651903 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0514cb19-b6bc-4e1c-8c3a-5003e1f26297 admin admin] The token cache is being invalidate because project ff9c9354e0164e55ae92308199240cb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:02.652487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0514cb19-b6bc-4e1c-8c3a-5003e1f26297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:02.653795 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0514cb19-b6bc-4e1c-8c3a-5003e1f26297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:02.656211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 978/1969] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:02 2022] DELETE /identity/v3/projects/ff9c9354e0164e55ae92308199240cb7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:02.661750 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213b565-395f-4333-9e80-f6bc27937f8f None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.661930 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213b565-395f-4333-9e80-f6bc27937f8f None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.662312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213b565-395f-4333-9e80-f6bc27937f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.682729 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-3213b565-395f-4333-9e80-f6bc27937f8f None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:02.684027 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-3213b565-395f-4333-9e80-f6bc27937f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:02.685271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 992/1970] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:18:02.692552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 979/1971] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:18:02.698055 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3d4ce-1804-4cca-95ce-0da346353dbf None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:02.699516 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1fa3d4ce-1804-4cca-95ce-0da346353dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:02.701650 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dea2cf-e135-4712-bec1-6813e773bd80 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.701942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dea2cf-e135-4712-bec1-6813e773bd80 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.702305 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dea2cf-e135-4712-bec1-6813e773bd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.722056 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-46dea2cf-e135-4712-bec1-6813e773bd80 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:02.723366 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-46dea2cf-e135-4712-bec1-6813e773bd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:02.731698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 980/1972] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:18:02.735898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dea2cf-e135-4712-bec1-6813e773bd80 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:02.746046 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dea2cf-e135-4712-bec1-6813e773bd80 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:02.751450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc717cb-ef70-4116-bab4-3c2558fa735d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:02.752052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc717cb-ef70-4116-bab4-3c2558fa735d admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.752328 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc717cb-ef70-4116-bab4-3c2558fa735d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.752665 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc717cb-ef70-4116-bab4-3c2558fa735d admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.753358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc717cb-ef70-4116-bab4-3c2558fa735d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:18:02.753692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc717cb-ef70-4116-bab4-3c2558fa735d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:02.755312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc717cb-ef70-4116-bab4-3c2558fa735d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:02.756725 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa3d4ce-1804-4cca-95ce-0da346353dbf None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:02.761594 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9de32a-cf50-4ab2-a8c8-0d1dfb2a8783 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:02.762128 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9de32a-cf50-4ab2-a8c8-0d1dfb2a8783 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.762410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9de32a-cf50-4ab2-a8c8-0d1dfb2a8783 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.763178 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9de32a-cf50-4ab2-a8c8-0d1dfb2a8783 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.767866 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 981/1973] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:18:02.772133 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc717cb-ef70-4116-bab4-3c2558fa735d admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:02.772928 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9de32a-cf50-4ab2-a8c8-0d1dfb2a8783 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:02.774357 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9de32a-cf50-4ab2-a8c8-0d1dfb2a8783 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:02.783375 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc717cb-ef70-4116-bab4-3c2558fa735d admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:02.784517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 993/1974] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:02.788183 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d0868-e0a3-4eeb-93a4-257021c35b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:02.788741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d0868-e0a3-4eeb-93a4-257021c35b84 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.789020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d0868-e0a3-4eeb-93a4-257021c35b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.789289 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795d0868-e0a3-4eeb-93a4-257021c35b84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.789946 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d0868-e0a3-4eeb-93a4-257021c35b84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:02.791820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795d0868-e0a3-4eeb-93a4-257021c35b84 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:02.817480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:02.829661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 982/1975] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:18:02 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:18:02.834040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9de32a-cf50-4ab2-a8c8-0d1dfb2a8783 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:02.844599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9de32a-cf50-4ab2-a8c8-0d1dfb2a8783 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:02.906839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f93a4b-d150-48bb-9328-5ae4d99bac04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:02.907474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f93a4b-d150-48bb-9328-5ae4d99bac04 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.907753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f93a4b-d150-48bb-9328-5ae4d99bac04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.908129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f93a4b-d150-48bb-9328-5ae4d99bac04 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.908755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f93a4b-d150-48bb-9328-5ae4d99bac04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:02.910781 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f93a4b-d150-48bb-9328-5ae4d99bac04 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:02.934247 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 994/1976] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:18:02 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:18:02.938424 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d0868-e0a3-4eeb-93a4-257021c35b84 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:02.950618 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795d0868-e0a3-4eeb-93a4-257021c35b84 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:02.955749 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8276d-81c4-4264-8d61-48834bd1348e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:02.956358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8276d-81c4-4264-8d61-48834bd1348e admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.956662 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8276d-81c4-4264-8d61-48834bd1348e admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.956941 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8276d-81c4-4264-8d61-48834bd1348e admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.957959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8276d-81c4-4264-8d61-48834bd1348e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:02.959572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e8276d-81c4-4264-8d61-48834bd1348e admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:02.972681 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 983/1977] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:02.974423 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f93a4b-d150-48bb-9328-5ae4d99bac04 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:02.986546 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f93a4b-d150-48bb-9328-5ae4d99bac04 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:02.991492 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fcf849-82f8-4468-bbaa-9d0470df04ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:02.992713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcf849-82f8-4468-bbaa-9d0470df04ee admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:02.992993 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcf849-82f8-4468-bbaa-9d0470df04ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:02.993263 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fcf849-82f8-4468-bbaa-9d0470df04ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:02.993958 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fcf849-82f8-4468-bbaa-9d0470df04ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:02.995550 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fcf849-82f8-4468-bbaa-9d0470df04ee admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:03.006850 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 995/1978] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:03.012277 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8276d-81c4-4264-8d61-48834bd1348e admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:03.022821 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8276d-81c4-4264-8d61-48834bd1348e admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:03.027555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef85652-6164-4791-acd6-3531eb81bcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:03.028228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef85652-6164-4791-acd6-3531eb81bcf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:03.028511 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef85652-6164-4791-acd6-3531eb81bcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:03.028842 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef85652-6164-4791-acd6-3531eb81bcf5 admin admin] PATH_INFO: `/v3/projects/69bc670511d847b980a77302686d3717/users/7c4eaad01b9040fbb56c346ee7400396/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:03.045833 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef85652-6164-4791-acd6-3531eb81bcf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69bc670511d847b980a77302686d3717, user_id=7c4eaad01b9040fbb56c346ee7400396, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:03.048562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef85652-6164-4791-acd6-3531eb81bcf5 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:03.063203 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 984/1979] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:03 2022] PUT /identity/v3/projects/69bc670511d847b980a77302686d3717/users/7c4eaad01b9040fbb56c346ee7400396/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:03.068726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefa81e-525f-4f39-8bcd-4fafc4f3e612 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:03.069224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefa81e-525f-4f39-8bcd-4fafc4f3e612 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:03.069490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbefa81e-525f-4f39-8bcd-4fafc4f3e612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:03.091663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-fbefa81e-525f-4f39-8bcd-4fafc4f3e612 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:03.093516 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-fbefa81e-525f-4f39-8bcd-4fafc4f3e612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:03.217809 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:03.239164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 996/1980] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:03.249404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef85652-6164-4791-acd6-3531eb81bcf5 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:03.325962 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef85652-6164-4791-acd6-3531eb81bcf5 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:03.329617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b901ae-4f44-4625-840c-0f70915f6891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:03.330241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b901ae-4f44-4625-840c-0f70915f6891 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:03.330525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b901ae-4f44-4625-840c-0f70915f6891 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:03.330797 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b901ae-4f44-4625-840c-0f70915f6891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:03.342138 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b901ae-4f44-4625-840c-0f70915f6891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:03.343665 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b901ae-4f44-4625-840c-0f70915f6891 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:03.353890 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 985/1981] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:04.037707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0965d-42cf-4841-aac6-c4eaa68ef73c None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:04.037998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0965d-42cf-4841-aac6-c4eaa68ef73c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:04.038276 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b0965d-42cf-4841-aac6-c4eaa68ef73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:04.068128 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-98b0965d-42cf-4841-aac6-c4eaa68ef73c None None] MFA Rules not processed for user `7c4eaad01b9040fbb56c346ee7400396`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:04.069335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-98b0965d-42cf-4841-aac6-c4eaa68ef73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:04.146902 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 997/1982] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:04.151931 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b901ae-4f44-4625-840c-0f70915f6891 service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:04.166688 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b901ae-4f44-4625-840c-0f70915f6891 service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:04.171146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5db083-a61d-47ca-8e7f-db1c790daa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:04.171837 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5db083-a61d-47ca-8e7f-db1c790daa8d admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:04.172221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5db083-a61d-47ca-8e7f-db1c790daa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:04.172987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5db083-a61d-47ca-8e7f-db1c790daa8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:04.173439 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5db083-a61d-47ca-8e7f-db1c790daa8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:04.175594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5db083-a61d-47ca-8e7f-db1c790daa8d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:04.211496 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 986/1983] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:18:04 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:18:04.216719 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0965d-42cf-4841-aac6-c4eaa68ef73c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:04.233938 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b0965d-42cf-4841-aac6-c4eaa68ef73c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:04.323545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648cb67-9523-45f8-b881-417d1a1e380d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:04.324386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648cb67-9523-45f8-b881-417d1a1e380d admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:04.325033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648cb67-9523-45f8-b881-417d1a1e380d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:04.325442 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648cb67-9523-45f8-b881-417d1a1e380d admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:04.326315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648cb67-9523-45f8-b881-417d1a1e380d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:04.329221 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648cb67-9523-45f8-b881-417d1a1e380d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:04.365426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 998/1984] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:18:04 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:04.366669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5db083-a61d-47ca-8e7f-db1c790daa8d admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:04.379501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5db083-a61d-47ca-8e7f-db1c790daa8d admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:04.385717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247a53a-986c-4605-bc0f-cd77b7caee9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:04.386342 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247a53a-986c-4605-bc0f-cd77b7caee9d admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:04.386726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247a53a-986c-4605-bc0f-cd77b7caee9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:04.386887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b247a53a-986c-4605-bc0f-cd77b7caee9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:04.387777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247a53a-986c-4605-bc0f-cd77b7caee9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:04.389436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b247a53a-986c-4605-bc0f-cd77b7caee9d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:04.406388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 987/1985] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:04.407339 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648cb67-9523-45f8-b881-417d1a1e380d admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:04.432958 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648cb67-9523-45f8-b881-417d1a1e380d admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:04.443236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaee4cb-2b0e-4294-942b-7b54f43c29f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:04.446021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaee4cb-2b0e-4294-942b-7b54f43c29f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:04.446498 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaee4cb-2b0e-4294-942b-7b54f43c29f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:04.446874 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaee4cb-2b0e-4294-942b-7b54f43c29f4 admin admin] PATH_INFO: `/v3/projects/639e766fba9d4f358d8afb03e939e065/users/e506d3a50a394895b5ae54b1455b797a/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:04.476379 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaee4cb-2b0e-4294-942b-7b54f43c29f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=639e766fba9d4f358d8afb03e939e065, user_id=e506d3a50a394895b5ae54b1455b797a, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:04.479469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaee4cb-2b0e-4294-942b-7b54f43c29f4 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:04.496295 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 999/1986] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:04 2022] PUT /identity/v3/projects/639e766fba9d4f358d8afb03e939e065/users/e506d3a50a394895b5ae54b1455b797a/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:04.500874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247a53a-986c-4605-bc0f-cd77b7caee9d admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:04.527266 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b247a53a-986c-4605-bc0f-cd77b7caee9d admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:04.606394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcf609f-d5f6-45a7-b7f7-3e9ae3898531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:04.606989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf609f-d5f6-45a7-b7f7-3e9ae3898531 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:04.607269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf609f-d5f6-45a7-b7f7-3e9ae3898531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:04.607539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcf609f-d5f6-45a7-b7f7-3e9ae3898531 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:04.608263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf609f-d5f6-45a7-b7f7-3e9ae3898531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:04.609936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcf609f-d5f6-45a7-b7f7-3e9ae3898531 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:04.628374 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 988/1987] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:04.630122 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaee4cb-2b0e-4294-942b-7b54f43c29f4 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:04.642441 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaee4cb-2b0e-4294-942b-7b54f43c29f4 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:04.647387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bf041-b722-48cd-a718-cb65d429eabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:04.648817 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bf041-b722-48cd-a718-cb65d429eabf admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:04.649154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bf041-b722-48cd-a718-cb65d429eabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:04.649388 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bf041-b722-48cd-a718-cb65d429eabf admin admin] PATH_INFO: `/v3/projects/639e766fba9d4f358d8afb03e939e065/users/e506d3a50a394895b5ae54b1455b797a/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:04.651685 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31bf041-b722-48cd-a718-cb65d429eabf admin admin] RBAC: Authorizing `identity:create_grant(project_id=639e766fba9d4f358d8afb03e939e065, user_id=e506d3a50a394895b5ae54b1455b797a, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:04.654584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31bf041-b722-48cd-a718-cb65d429eabf admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:04.668845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1000/1988] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:04 2022] PUT /identity/v3/projects/639e766fba9d4f358d8afb03e939e065/users/e506d3a50a394895b5ae54b1455b797a/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:05.983161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee815b-5e5d-494c-8491-f3e552c2dccf None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:05.983696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee815b-5e5d-494c-8491-f3e552c2dccf None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:05.983784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bee815b-5e5d-494c-8491-f3e552c2dccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:06.045612 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-4bee815b-5e5d-494c-8491-f3e552c2dccf None None] MFA Rules not processed for user `e506d3a50a394895b5ae54b1455b797a`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:06.047190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-4bee815b-5e5d-494c-8491-f3e552c2dccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:06.070422 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:06.156664 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 989/1989] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:06.163507 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bf041-b722-48cd-a718-cb65d429eabf admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:06.165824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a31bf041-b722-48cd-a718-cb65d429eabf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:06.228180 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:06.255925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31bf041-b722-48cd-a718-cb65d429eabf admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:06.260241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfe75f-5241-4cd1-bc86-a87078229354 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:06.261065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfe75f-5241-4cd1-bc86-a87078229354 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:06.261418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfe75f-5241-4cd1-bc86-a87078229354 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:06.261764 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfe75f-5241-4cd1-bc86-a87078229354 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:06.328168 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedfe75f-5241-4cd1-bc86-a87078229354 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:06.329771 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedfe75f-5241-4cd1-bc86-a87078229354 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:06.345994 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1001/1990] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:08.048073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee815b-5e5d-494c-8491-f3e552c2dccf None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:08.062202 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bee815b-5e5d-494c-8491-f3e552c2dccf None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:08.067195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462acbf6-9080-442a-bf03-0f8ecb04d423 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:08.068082 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462acbf6-9080-442a-bf03-0f8ecb04d423 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:08.068522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462acbf6-9080-442a-bf03-0f8ecb04d423 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:08.069019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462acbf6-9080-442a-bf03-0f8ecb04d423 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:08.079311 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462acbf6-9080-442a-bf03-0f8ecb04d423 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:08.081465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462acbf6-9080-442a-bf03-0f8ecb04d423 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:08.106224 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 990/1991] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:08.127150 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b23a69-cdd8-4eb8-a75b-a8a49f535fda None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:08.128233 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b23a69-cdd8-4eb8-a75b-a8a49f535fda None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:08.128885 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b23a69-cdd8-4eb8-a75b-a8a49f535fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:08.167276 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-01b23a69-cdd8-4eb8-a75b-a8a49f535fda None None] MFA Rules not processed for user `11bf0abff172401bb62653e975c91f45`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:08.168524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-01b23a69-cdd8-4eb8-a75b-a8a49f535fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:08.246834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1002/1992] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:08.254553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462acbf6-9080-442a-bf03-0f8ecb04d423 service cinder] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:08.266055 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-462acbf6-9080-442a-bf03-0f8ecb04d423 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:08.352983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462acbf6-9080-442a-bf03-0f8ecb04d423 service cinder] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:08.364466 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9aabb-3853-48bf-bef1-512269ef245d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:08.365188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9aabb-3853-48bf-bef1-512269ef245d admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:08.365525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9aabb-3853-48bf-bef1-512269ef245d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:08.365853 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9aabb-3853-48bf-bef1-512269ef245d admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:08.366770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9aabb-3853-48bf-bef1-512269ef245d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:08.370584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9aabb-3853-48bf-bef1-512269ef245d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:08.410329 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 991/1993] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:18:08 2022] POST /identity/v3/projects => generated 371 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:08.414696 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b23a69-cdd8-4eb8-a75b-a8a49f535fda None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:08.427345 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b23a69-cdd8-4eb8-a75b-a8a49f535fda None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:08.504225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d080fef-2664-4681-aee3-e399a707172d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:08.504999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d080fef-2664-4681-aee3-e399a707172d admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:08.505350 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d080fef-2664-4681-aee3-e399a707172d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:08.505689 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d080fef-2664-4681-aee3-e399a707172d admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:08.506466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d080fef-2664-4681-aee3-e399a707172d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:08.508560 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d080fef-2664-4681-aee3-e399a707172d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:08.531949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1003/1994] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:18:08 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:08.535834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9aabb-3853-48bf-bef1-512269ef245d admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:08.547515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9aabb-3853-48bf-bef1-512269ef245d admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:08.553367 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad653b-8596-435b-8d28-c7ecaa701af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:08.553981 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad653b-8596-435b-8d28-c7ecaa701af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:08.554356 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad653b-8596-435b-8d28-c7ecaa701af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:08.554634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad653b-8596-435b-8d28-c7ecaa701af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:08.555345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad653b-8596-435b-8d28-c7ecaa701af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:08.556955 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad653b-8596-435b-8d28-c7ecaa701af3 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:08.568466 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 992/1995] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:08.572577 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d080fef-2664-4681-aee3-e399a707172d admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:08.584176 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d080fef-2664-4681-aee3-e399a707172d admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:08.589378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab33c36-3ceb-4987-b850-114b10d436d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:08.590053 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab33c36-3ceb-4987-b850-114b10d436d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:08.590329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab33c36-3ceb-4987-b850-114b10d436d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:08.590598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab33c36-3ceb-4987-b850-114b10d436d2 admin admin] PATH_INFO: `/v3/projects/14a7085aaa61410885bb0d39cb6d80b0/users/56ee73ed8e244a4e8a434c4cd7f732e9/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:08.594076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:08.609577 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab33c36-3ceb-4987-b850-114b10d436d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a7085aaa61410885bb0d39cb6d80b0, user_id=56ee73ed8e244a4e8a434c4cd7f732e9, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:08.612307 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab33c36-3ceb-4987-b850-114b10d436d2 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:08.627227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1004/1996] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:08 2022] PUT /identity/v3/projects/14a7085aaa61410885bb0d39cb6d80b0/users/56ee73ed8e244a4e8a434c4cd7f732e9/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:08.632035 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad653b-8596-435b-8d28-c7ecaa701af3 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:08.643663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad653b-8596-435b-8d28-c7ecaa701af3 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:08.721039 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015f1b02-7ee5-4e9b-980e-5f034715f4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:08.721741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f1b02-7ee5-4e9b-980e-5f034715f4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:08.722259 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f1b02-7ee5-4e9b-980e-5f034715f4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:08.722609 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015f1b02-7ee5-4e9b-980e-5f034715f4b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:08.723396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f1b02-7ee5-4e9b-980e-5f034715f4b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:08.725163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015f1b02-7ee5-4e9b-980e-5f034715f4b0 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:08.737807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 993/1997] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:08.741828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab33c36-3ceb-4987-b850-114b10d436d2 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:08.752590 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab33c36-3ceb-4987-b850-114b10d436d2 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:08.757740 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341f351-eb2e-451e-bc1e-6a7adf210ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:08.758585 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341f351-eb2e-451e-bc1e-6a7adf210ac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:08.758872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341f351-eb2e-451e-bc1e-6a7adf210ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:08.759149 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341f351-eb2e-451e-bc1e-6a7adf210ac0 admin admin] PATH_INFO: `/v3/projects/14a7085aaa61410885bb0d39cb6d80b0/users/56ee73ed8e244a4e8a434c4cd7f732e9/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:08.761176 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341f351-eb2e-451e-bc1e-6a7adf210ac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a7085aaa61410885bb0d39cb6d80b0, user_id=56ee73ed8e244a4e8a434c4cd7f732e9, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:08.763832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341f351-eb2e-451e-bc1e-6a7adf210ac0 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:08.775122 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1005/1998] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:08 2022] PUT /identity/v3/projects/14a7085aaa61410885bb0d39cb6d80b0/users/56ee73ed8e244a4e8a434c4cd7f732e9/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:14.389366 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacdeab-373a-405c-8a4a-dbce3e8cf68f None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:14.389828 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacdeab-373a-405c-8a4a-dbce3e8cf68f None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:14.390510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacdeab-373a-405c-8a4a-dbce3e8cf68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:14.455342 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:14.487282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-ecacdeab-373a-405c-8a4a-dbce3e8cf68f None None] MFA Rules not processed for user `56ee73ed8e244a4e8a434c4cd7f732e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:14.488775 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ecacdeab-373a-405c-8a4a-dbce3e8cf68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:14.680899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 994/1999] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:14 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:14.690195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341f351-eb2e-451e-bc1e-6a7adf210ac0 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:14.692508 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a341f351-eb2e-451e-bc1e-6a7adf210ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:14.910426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341f351-eb2e-451e-bc1e-6a7adf210ac0 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:14.918063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066546d-7a89-48e2-b61f-36fb2f1aca6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:14.918736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066546d-7a89-48e2-b61f-36fb2f1aca6e service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:14.919020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066546d-7a89-48e2-b61f-36fb2f1aca6e service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:14.919301 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d066546d-7a89-48e2-b61f-36fb2f1aca6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:15.002500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066546d-7a89-48e2-b61f-36fb2f1aca6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:15.004928 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d066546d-7a89-48e2-b61f-36fb2f1aca6e service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:15.019557 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1006/2000] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:16.369876 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb91a1-475f-46ae-a7d4-9e8945b8e50d None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:16.370376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb91a1-475f-46ae-a7d4-9e8945b8e50d None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:16.370468 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb91a1-475f-46ae-a7d4-9e8945b8e50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:16.392314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-eacb91a1-475f-46ae-a7d4-9e8945b8e50d None None] MFA Rules not processed for user `11bf0abff172401bb62653e975c91f45`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:16.393591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-eacb91a1-475f-46ae-a7d4-9e8945b8e50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:16.400917 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 995/2001] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:23.218799 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066546d-7a89-48e2-b61f-36fb2f1aca6e service nova] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:23.220862 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d066546d-7a89-48e2-b61f-36fb2f1aca6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:23.381162 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d066546d-7a89-48e2-b61f-36fb2f1aca6e service nova] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:23.389398 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6901da-5f33-4855-9eb9-b17d8ded0a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:23.392218 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6901da-5f33-4855-9eb9-b17d8ded0a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:23.392505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6901da-5f33-4855-9eb9-b17d8ded0a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:23.392964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6901da-5f33-4855-9eb9-b17d8ded0a48 admin admin] PATH_INFO: `/v3/users/9a927cd4c01d4fc3ac0752c2b14b1994` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:23.394329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6901da-5f33-4855-9eb9-b17d8ded0a48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a927cd4c01d4fc3ac0752c2b14b1994)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:23.396971 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6901da-5f33-4855-9eb9-b17d8ded0a48 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:23.416938 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:23.477122 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6b6901da-5f33-4855-9eb9-b17d8ded0a48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a927cd4c01d4fc3ac0752c2b14b1994'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:23.486081 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6b6901da-5f33-4855-9eb9-b17d8ded0a48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a927cd4c01d4fc3ac0752c2b14b1994'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:23.486848 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6b6901da-5f33-4855-9eb9-b17d8ded0a48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a927cd4c01d4fc3ac0752c2b14b1994'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:23.501599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6b6901da-5f33-4855-9eb9-b17d8ded0a48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a927cd4c01d4fc3ac0752c2b14b1994'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:23.502419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6b6901da-5f33-4855-9eb9-b17d8ded0a48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a927cd4c01d4fc3ac0752c2b14b1994'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:23.521048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1007/2002] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:18:23 2022] DELETE /identity/v3/users/9a927cd4c01d4fc3ac0752c2b14b1994 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:23.526395 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec95a9-f7f8-44c2-84fe-13327b11bc4a None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:23.526978 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec95a9-f7f8-44c2-84fe-13327b11bc4a None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:23.526978 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec95a9-f7f8-44c2-84fe-13327b11bc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:23.550195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-47ec95a9-f7f8-44c2-84fe-13327b11bc4a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:23.551551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-47ec95a9-f7f8-44c2-84fe-13327b11bc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:23.678523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 996/2003] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:18:23.692194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6901da-5f33-4855-9eb9-b17d8ded0a48 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:23.693875 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-6b6901da-5f33-4855-9eb9-b17d8ded0a48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:23.762838 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6901da-5f33-4855-9eb9-b17d8ded0a48 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:23.766441 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf1557-0f1f-43d6-b85b-ef317afd6062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:23.767063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf1557-0f1f-43d6-b85b-ef317afd6062 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:23.767342 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf1557-0f1f-43d6-b85b-ef317afd6062 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:23.767611 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf1557-0f1f-43d6-b85b-ef317afd6062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:23.777401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf1557-0f1f-43d6-b85b-ef317afd6062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:23.778825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf1557-0f1f-43d6-b85b-ef317afd6062 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:23.788890 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1008/2004] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:24.204607 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec95a9-f7f8-44c2-84fe-13327b11bc4a None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:24.206162 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-47ec95a9-f7f8-44c2-84fe-13327b11bc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:24.218937 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec95a9-f7f8-44c2-84fe-13327b11bc4a None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:24.236516 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa85c89-d38a-40a3-8b32-11b66e4fd535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:24.237329 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa85c89-d38a-40a3-8b32-11b66e4fd535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:24.238555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa85c89-d38a-40a3-8b32-11b66e4fd535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:24.238903 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa85c89-d38a-40a3-8b32-11b66e4fd535 admin admin] PATH_INFO: `/v3/projects/6f2c985de2e84620a80a708ce6c8240e` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:24.240465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa85c89-d38a-40a3-8b32-11b66e4fd535 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f2c985de2e84620a80a708ce6c8240e)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:24.243328 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa85c89-d38a-40a3-8b32-11b66e4fd535 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:24.321573 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-afa85c89-d38a-40a3-8b32-11b66e4fd535 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f2c985de2e84620a80a708ce6c8240e', 'request_id': 'req-afa85c89-d38a-40a3-8b32-11b66e4fd535'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:24.335232 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-afa85c89-d38a-40a3-8b32-11b66e4fd535 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f2c985de2e84620a80a708ce6c8240e', 'request_id': 'req-afa85c89-d38a-40a3-8b32-11b66e4fd535'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:24.346154 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-afa85c89-d38a-40a3-8b32-11b66e4fd535 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f2c985de2e84620a80a708ce6c8240e', 'request_id': 'req-afa85c89-d38a-40a3-8b32-11b66e4fd535'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:24.360079 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-afa85c89-d38a-40a3-8b32-11b66e4fd535 admin admin] The token cache is being invalidate because project 6f2c985de2e84620a80a708ce6c8240e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:24.360368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-afa85c89-d38a-40a3-8b32-11b66e4fd535 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:24.361089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-afa85c89-d38a-40a3-8b32-11b66e4fd535 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:24.363092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 997/2005] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:24 2022] DELETE /identity/v3/projects/6f2c985de2e84620a80a708ce6c8240e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:24.368808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675852f-3175-41b7-b53f-7accaf39874e None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:24.369297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675852f-3175-41b7-b53f-7accaf39874e None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:24.369387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675852f-3175-41b7-b53f-7accaf39874e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:24.397122 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-4675852f-3175-41b7-b53f-7accaf39874e None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:24.399463 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4675852f-3175-41b7-b53f-7accaf39874e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:24.551791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1009/2006] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:18:24.552823 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce7c03-2142-4990-972c-ceca78b5c79c None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:24.553275 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce7c03-2142-4990-972c-ceca78b5c79c None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:24.553609 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce7c03-2142-4990-972c-ceca78b5c79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:24.586351 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-d4ce7c03-2142-4990-972c-ceca78b5c79c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:24.589536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d4ce7c03-2142-4990-972c-ceca78b5c79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:24.605623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 998/2007] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:24.609715 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675852f-3175-41b7-b53f-7accaf39874e None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:24.625524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675852f-3175-41b7-b53f-7accaf39874e None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:24.631542 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdcac5-ab62-43ad-b172-f60550fcf88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:24.633568 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdcac5-ab62-43ad-b172-f60550fcf88d admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:24.634012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdcac5-ab62-43ad-b172-f60550fcf88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:24.634289 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdcac5-ab62-43ad-b172-f60550fcf88d admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:24.635050 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdcac5-ab62-43ad-b172-f60550fcf88d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:18:24.635354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdcac5-ab62-43ad-b172-f60550fcf88d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:24.637033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cdcac5-ab62-43ad-b172-f60550fcf88d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:24.655529 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1010/2008] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:18:24.656938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce7c03-2142-4990-972c-ceca78b5c79c None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:24.668061 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce7c03-2142-4990-972c-ceca78b5c79c None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:24.681047 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfec0e7-f42c-4d79-8842-d4ea03f2b7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:24.681882 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfec0e7-f42c-4d79-8842-d4ea03f2b7b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:24.682309 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfec0e7-f42c-4d79-8842-d4ea03f2b7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:24.682725 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfec0e7-f42c-4d79-8842-d4ea03f2b7b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:24.683564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfec0e7-f42c-4d79-8842-d4ea03f2b7b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:24.702945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfec0e7-f42c-4d79-8842-d4ea03f2b7b6 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:24.716686 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:24.771031 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 999/2009] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:18:24 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:24.775451 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdcac5-ab62-43ad-b172-f60550fcf88d admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:24.787282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdcac5-ab62-43ad-b172-f60550fcf88d admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:24.810278 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:24.891272 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43143272-5e12-4a3d-8afa-aebbff4ea12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:24.891867 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43143272-5e12-4a3d-8afa-aebbff4ea12d admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:24.892857 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43143272-5e12-4a3d-8afa-aebbff4ea12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:24.892857 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43143272-5e12-4a3d-8afa-aebbff4ea12d admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:24.893573 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43143272-5e12-4a3d-8afa-aebbff4ea12d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:24.895920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43143272-5e12-4a3d-8afa-aebbff4ea12d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:24.938398 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1011/2010] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:18:24 2022] POST /identity/v3/users => generated 294 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:24.940243 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfec0e7-f42c-4d79-8842-d4ea03f2b7b6 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:24.953007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfec0e7-f42c-4d79-8842-d4ea03f2b7b6 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:24.963937 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1efb104-87dd-47fb-8ccc-b469603c21c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:24.965451 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efb104-87dd-47fb-8ccc-b469603c21c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:24.965886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efb104-87dd-47fb-8ccc-b469603c21c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:24.966311 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1efb104-87dd-47fb-8ccc-b469603c21c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:24.967188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1efb104-87dd-47fb-8ccc-b469603c21c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:24.969199 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1efb104-87dd-47fb-8ccc-b469603c21c1 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:24.987778 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1000/2011] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:24.990127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43143272-5e12-4a3d-8afa-aebbff4ea12d admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:25.003324 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43143272-5e12-4a3d-8afa-aebbff4ea12d admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:25.007177 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cee0fc-03d1-4944-8194-bc12022ea149 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:25.007508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cee0fc-03d1-4944-8194-bc12022ea149 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:25.007784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cee0fc-03d1-4944-8194-bc12022ea149 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:25.009699 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1001/2012] 10.222.0.45 () {54 vars in 921 bytes} [Thu Aug 4 02:18:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:18:25.013351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaea622-89cb-49db-88aa-9ff2b655ff8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:25.013990 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaea622-89cb-49db-88aa-9ff2b655ff8c admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:25.014275 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaea622-89cb-49db-88aa-9ff2b655ff8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:25.014537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaea622-89cb-49db-88aa-9ff2b655ff8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:25.015315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaea622-89cb-49db-88aa-9ff2b655ff8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:25.017011 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaea622-89cb-49db-88aa-9ff2b655ff8c admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:25.037377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cee0fc-03d1-4944-8194-bc12022ea149 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:25.037821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1012/2013] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:25.051594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cee0fc-03d1-4944-8194-bc12022ea149 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:25.059613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29806f18-9b38-494b-b97e-563e3a7cb354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:25.059613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29806f18-9b38-494b-b97e-563e3a7cb354 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:25.059613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29806f18-9b38-494b-b97e-563e3a7cb354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:25.059613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29806f18-9b38-494b-b97e-563e3a7cb354 admin admin] PATH_INFO: `/v3/projects/b5150eed305142bc9cf3a5dfb48d2583/users/d4a9fa95ae1245a3b705d3d181146451/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:25.061798 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230be458-077a-40c3-a544-ee00e8773e86 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:25.063299 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230be458-077a-40c3-a544-ee00e8773e86 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:25.063749 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230be458-077a-40c3-a544-ee00e8773e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:25.077562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29806f18-9b38-494b-b97e-563e3a7cb354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5150eed305142bc9cf3a5dfb48d2583, user_id=d4a9fa95ae1245a3b705d3d181146451, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:25.082105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29806f18-9b38-494b-b97e-563e3a7cb354 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:25.093674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-230be458-077a-40c3-a544-ee00e8773e86 None None] MFA Rules not processed for user `796772cf491047c8b0a609dae9402307`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:25.095049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-230be458-077a-40c3-a544-ee00e8773e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:25.105249 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1002/2014] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:25 2022] PUT /identity/v3/projects/b5150eed305142bc9cf3a5dfb48d2583/users/d4a9fa95ae1245a3b705d3d181146451/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:25.120259 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5adcd-9d87-470d-8002-fbf9093dbbd0 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:25.120557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5adcd-9d87-470d-8002-fbf9093dbbd0 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:25.120874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5adcd-9d87-470d-8002-fbf9093dbbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:25.161746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-0ba5adcd-9d87-470d-8002-fbf9093dbbd0 None None] MFA Rules not processed for user `d4a9fa95ae1245a3b705d3d181146451`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:25.163230 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-0ba5adcd-9d87-470d-8002-fbf9093dbbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:25.203923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1013/2015] 10.222.0.45 () {58 vars in 1018 bytes} [Thu Aug 4 02:18:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:18:25.204879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230be458-077a-40c3-a544-ee00e8773e86 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:25.206296 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-230be458-077a-40c3-a544-ee00e8773e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:25.259681 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1003/2016] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:25 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:25.261312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5adcd-9d87-470d-8002-fbf9093dbbd0 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:25.262652 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-0ba5adcd-9d87-470d-8002-fbf9093dbbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:25.285379 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5adcd-9d87-470d-8002-fbf9093dbbd0 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:25.293806 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62026b-e62e-4797-86b6-15984ed0505c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796772cf491047c8b0a609dae9402307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:25.298277 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62026b-e62e-4797-86b6-15984ed0505c service placement] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:25.298745 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62026b-e62e-4797-86b6-15984ed0505c service placement] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:25.300809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62026b-e62e-4797-86b6-15984ed0505c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:25.301636 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230be458-077a-40c3-a544-ee00e8773e86 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:25.307754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d7892-964a-431e-8c58-8cacc8aded98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:25.308507 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d7892-964a-431e-8c58-8cacc8aded98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:25.308884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d7892-964a-431e-8c58-8cacc8aded98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:25.309321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59d7892-964a-431e-8c58-8cacc8aded98 admin admin] PATH_INFO: `/v3/users/44655d95949b489088dbe48eeb8f38f7` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:25.310783 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d7892-964a-431e-8c58-8cacc8aded98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44655d95949b489088dbe48eeb8f38f7)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:25.313374 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59d7892-964a-431e-8c58-8cacc8aded98 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:25.317117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62026b-e62e-4797-86b6-15984ed0505c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:25.320987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62026b-e62e-4797-86b6-15984ed0505c service placement] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:25.336316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1004/2017] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:25.337429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62026b-e62e-4797-86b6-15984ed0505c service placement] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:25.338949 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-dc62026b-e62e-4797-86b6-15984ed0505c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:25.379454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:25.410795 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d59d7892-964a-431e-8c58-8cacc8aded98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44655d95949b489088dbe48eeb8f38f7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:25.422023 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d59d7892-964a-431e-8c58-8cacc8aded98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44655d95949b489088dbe48eeb8f38f7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:25.423126 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d59d7892-964a-431e-8c58-8cacc8aded98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44655d95949b489088dbe48eeb8f38f7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:25.438071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d59d7892-964a-431e-8c58-8cacc8aded98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44655d95949b489088dbe48eeb8f38f7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:25.438886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62026b-e62e-4797-86b6-15984ed0505c service placement] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:25.439004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d59d7892-964a-431e-8c58-8cacc8aded98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44655d95949b489088dbe48eeb8f38f7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:25.442947 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a71a62-a099-4c6c-a385-617671eb8c49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:25.443678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a71a62-a099-4c6c-a385-617671eb8c49 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:25.444213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a71a62-a099-4c6c-a385-617671eb8c49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:25.444427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a71a62-a099-4c6c-a385-617671eb8c49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:25.447203 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-f6a71a62-a099-4c6c-a385-617671eb8c49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:25.457418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1014/2018] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:18:25 2022] DELETE /identity/v3/users/44655d95949b489088dbe48eeb8f38f7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:25.461785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a71a62-a099-4c6c-a385-617671eb8c49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:25.464577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a71a62-a099-4c6c-a385-617671eb8c49 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:25.480851 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1005/2019] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:25.674796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d7892-964a-431e-8c58-8cacc8aded98 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:25.677325 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d59d7892-964a-431e-8c58-8cacc8aded98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:25.776125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59d7892-964a-431e-8c58-8cacc8aded98 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:25.781275 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbfbc3-4ca0-4332-a556-a6b914d9e042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:25.782010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbfbc3-4ca0-4332-a556-a6b914d9e042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:25.782374 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbfbc3-4ca0-4332-a556-a6b914d9e042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:25.782764 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbfbc3-4ca0-4332-a556-a6b914d9e042 admin admin] PATH_INFO: `/v3/projects/de98ffc315d4474c8b5d5370ba1fe97c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:25.784681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fbfbc3-4ca0-4332-a556-a6b914d9e042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de98ffc315d4474c8b5d5370ba1fe97c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:25.787304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fbfbc3-4ca0-4332-a556-a6b914d9e042 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:25.854978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d3fbfbc3-4ca0-4332-a556-a6b914d9e042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de98ffc315d4474c8b5d5370ba1fe97c', 'request_id': 'req-d3fbfbc3-4ca0-4332-a556-a6b914d9e042'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:25.865295 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d3fbfbc3-4ca0-4332-a556-a6b914d9e042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de98ffc315d4474c8b5d5370ba1fe97c', 'request_id': 'req-d3fbfbc3-4ca0-4332-a556-a6b914d9e042'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:25.873032 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d3fbfbc3-4ca0-4332-a556-a6b914d9e042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de98ffc315d4474c8b5d5370ba1fe97c', 'request_id': 'req-d3fbfbc3-4ca0-4332-a556-a6b914d9e042'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:25.884843 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d3fbfbc3-4ca0-4332-a556-a6b914d9e042 admin admin] The token cache is being invalidate because project de98ffc315d4474c8b5d5370ba1fe97c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:25.885138 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d3fbfbc3-4ca0-4332-a556-a6b914d9e042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:25.885845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d3fbfbc3-4ca0-4332-a556-a6b914d9e042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:25.887674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1015/2020] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:25 2022] DELETE /identity/v3/projects/de98ffc315d4474c8b5d5370ba1fe97c => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:18:25.892517 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9331343-a2da-4840-938c-384a9e1e2292 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:25.892983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9331343-a2da-4840-938c-384a9e1e2292 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:25.893101 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9331343-a2da-4840-938c-384a9e1e2292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:25.916891 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-f9331343-a2da-4840-938c-384a9e1e2292 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:25.918183 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-f9331343-a2da-4840-938c-384a9e1e2292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:25.957090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:25.998830 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1006/2021] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:18:26.007397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609b73f-255e-443a-a582-b7637b21fd94 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:26.007588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609b73f-255e-443a-a582-b7637b21fd94 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:26.007847 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609b73f-255e-443a-a582-b7637b21fd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:26.030331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-f609b73f-255e-443a-a582-b7637b21fd94 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:26.031557 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f609b73f-255e-443a-a582-b7637b21fd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:26.040280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1016/2022] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:26.044758 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9331343-a2da-4840-938c-384a9e1e2292 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:26.055030 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9331343-a2da-4840-938c-384a9e1e2292 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:26.060283 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab4386-11e7-4a88-b3a9-9125aa277ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:26.060831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab4386-11e7-4a88-b3a9-9125aa277ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:26.061345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab4386-11e7-4a88-b3a9-9125aa277ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:26.061629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab4386-11e7-4a88-b3a9-9125aa277ce3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:26.062399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab4386-11e7-4a88-b3a9-9125aa277ce3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:18:26.062705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab4386-11e7-4a88-b3a9-9125aa277ce3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:26.064413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab4386-11e7-4a88-b3a9-9125aa277ce3 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:26.078808 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1007/2023] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:18:26.085190 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609b73f-255e-443a-a582-b7637b21fd94 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:26.095147 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609b73f-255e-443a-a582-b7637b21fd94 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:26.099777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef16ac0-48f2-473f-b60f-5695beecf687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:26.100392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef16ac0-48f2-473f-b60f-5695beecf687 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:26.100707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef16ac0-48f2-473f-b60f-5695beecf687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:26.101002 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef16ac0-48f2-473f-b60f-5695beecf687 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:26.102210 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef16ac0-48f2-473f-b60f-5695beecf687 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:26.104108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef16ac0-48f2-473f-b60f-5695beecf687 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:26.131058 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1017/2024] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:18:26 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:18:26.135501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab4386-11e7-4a88-b3a9-9125aa277ce3 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:26.145573 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab4386-11e7-4a88-b3a9-9125aa277ce3 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:26.204225 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb10fa-37b5-44ca-b3ed-6bb98701827b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:26.204910 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb10fa-37b5-44ca-b3ed-6bb98701827b admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:26.205194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb10fa-37b5-44ca-b3ed-6bb98701827b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:26.205479 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb10fa-37b5-44ca-b3ed-6bb98701827b admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:26.206129 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb10fa-37b5-44ca-b3ed-6bb98701827b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:26.208285 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb10fa-37b5-44ca-b3ed-6bb98701827b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:26.231095 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1008/2025] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:18:26 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:26.233467 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef16ac0-48f2-473f-b60f-5695beecf687 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:26.244304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef16ac0-48f2-473f-b60f-5695beecf687 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:26.249074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f5a9c-e813-4999-a25b-1ada9f0516f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:26.249749 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f5a9c-e813-4999-a25b-1ada9f0516f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:26.249990 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f5a9c-e813-4999-a25b-1ada9f0516f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:26.250266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f5a9c-e813-4999-a25b-1ada9f0516f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:26.251150 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f5a9c-e813-4999-a25b-1ada9f0516f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:26.252842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f5a9c-e813-4999-a25b-1ada9f0516f8 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:26.265895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1018/2026] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:26.269952 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb10fa-37b5-44ca-b3ed-6bb98701827b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:26.280898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb10fa-37b5-44ca-b3ed-6bb98701827b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:26.286285 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6aed16-8a34-4732-916f-15fcff4ed34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:26.286900 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aed16-8a34-4732-916f-15fcff4ed34f admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:26.287182 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aed16-8a34-4732-916f-15fcff4ed34f admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:26.287457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aed16-8a34-4732-916f-15fcff4ed34f admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:26.288156 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6aed16-8a34-4732-916f-15fcff4ed34f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:26.289929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6aed16-8a34-4732-916f-15fcff4ed34f admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:26.301562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1009/2027] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:18:26.305835 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f5a9c-e813-4999-a25b-1ada9f0516f8 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:26.311028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:26.317481 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f5a9c-e813-4999-a25b-1ada9f0516f8 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:26.323865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c4afc-f5d4-4b4b-a6b5-577f8b71c264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:26.324530 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c4afc-f5d4-4b4b-a6b5-577f8b71c264 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:26.324844 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c4afc-f5d4-4b4b-a6b5-577f8b71c264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:26.325125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7c4afc-f5d4-4b4b-a6b5-577f8b71c264 admin admin] PATH_INFO: `/v3/projects/df4b8fbecd0e4e25afe62d7d73a47f89/users/a55e354fcc69463693437b07890644ea/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:26.340507 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c4afc-f5d4-4b4b-a6b5-577f8b71c264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df4b8fbecd0e4e25afe62d7d73a47f89, user_id=a55e354fcc69463693437b07890644ea, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:26.343224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7c4afc-f5d4-4b4b-a6b5-577f8b71c264 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:26.358316 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1019/2028] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:26 2022] PUT /identity/v3/projects/df4b8fbecd0e4e25afe62d7d73a47f89/users/a55e354fcc69463693437b07890644ea/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:26.368574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2eb12f-efca-4e12-a760-24b920faa0ac None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:26.368815 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2eb12f-efca-4e12-a760-24b920faa0ac None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:26.369031 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2eb12f-efca-4e12-a760-24b920faa0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:26.404163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-1c2eb12f-efca-4e12-a760-24b920faa0ac None None] MFA Rules not processed for user `a55e354fcc69463693437b07890644ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:26.405571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-1c2eb12f-efca-4e12-a760-24b920faa0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:26.488659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1010/2029] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:26.497949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c4afc-f5d4-4b4b-a6b5-577f8b71c264 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:26.576670 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7c4afc-f5d4-4b4b-a6b5-577f8b71c264 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:26.581518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fb658-62da-459c-8bfd-3584ea7db7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:26.582144 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fb658-62da-459c-8bfd-3584ea7db7b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:26.582427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fb658-62da-459c-8bfd-3584ea7db7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:26.582703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0fb658-62da-459c-8bfd-3584ea7db7b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:26.583655 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fb658-62da-459c-8bfd-3584ea7db7b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:26.585762 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0fb658-62da-459c-8bfd-3584ea7db7b9 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:26.617802 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1020/2030] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:18:26 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:26.622191 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2eb12f-efca-4e12-a760-24b920faa0ac None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:26.633857 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2eb12f-efca-4e12-a760-24b920faa0ac None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:26.666606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:26.717483 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58d72a-14b5-4e04-903f-8fbd86d7293d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:26.718048 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58d72a-14b5-4e04-903f-8fbd86d7293d admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:26.718327 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58d72a-14b5-4e04-903f-8fbd86d7293d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:26.718690 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58d72a-14b5-4e04-903f-8fbd86d7293d admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:26.719363 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58d72a-14b5-4e04-903f-8fbd86d7293d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:26.721530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58d72a-14b5-4e04-903f-8fbd86d7293d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:26.758601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1011/2031] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:18:26 2022] POST /identity/v3/users => generated 297 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:26.762486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fb658-62da-459c-8bfd-3584ea7db7b9 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:26.773523 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0fb658-62da-459c-8bfd-3584ea7db7b9 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:26.778294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6d0b8-5ada-4cea-ab88-a01581624df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:26.779071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6d0b8-5ada-4cea-ab88-a01581624df7 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:26.779425 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6d0b8-5ada-4cea-ab88-a01581624df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:26.779770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb6d0b8-5ada-4cea-ab88-a01581624df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:26.780543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb6d0b8-5ada-4cea-ab88-a01581624df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:26.782258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb6d0b8-5ada-4cea-ab88-a01581624df7 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:26.794300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1021/2032] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:26.798520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58d72a-14b5-4e04-903f-8fbd86d7293d admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:26.809975 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58d72a-14b5-4e04-903f-8fbd86d7293d admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:26.814936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf754d-bd36-4998-ad9f-27d613c73be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:26.815622 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf754d-bd36-4998-ad9f-27d613c73be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:26.815904 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf754d-bd36-4998-ad9f-27d613c73be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:26.816180 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf754d-bd36-4998-ad9f-27d613c73be2 admin admin] PATH_INFO: `/v3/projects/bde03995f7744cd9b6493b5f4fbbe587/users/664503abad3148b3acb9a51af502f8d2/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:26.835666 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf754d-bd36-4998-ad9f-27d613c73be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde03995f7744cd9b6493b5f4fbbe587, user_id=664503abad3148b3acb9a51af502f8d2, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:26.838410 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf754d-bd36-4998-ad9f-27d613c73be2 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:26.853209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1012/2033] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:26 2022] PUT /identity/v3/projects/bde03995f7744cd9b6493b5f4fbbe587/users/664503abad3148b3acb9a51af502f8d2/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:26.857049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6d0b8-5ada-4cea-ab88-a01581624df7 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:26.867742 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb6d0b8-5ada-4cea-ab88-a01581624df7 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:26.935783 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09ff65-f7d9-4cfc-a578-98a020758215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:26.936423 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09ff65-f7d9-4cfc-a578-98a020758215 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:26.936730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09ff65-f7d9-4cfc-a578-98a020758215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:26.937009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09ff65-f7d9-4cfc-a578-98a020758215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:26.937784 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09ff65-f7d9-4cfc-a578-98a020758215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:26.939398 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09ff65-f7d9-4cfc-a578-98a020758215 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:26.950785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1022/2034] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:26.956012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf754d-bd36-4998-ad9f-27d613c73be2 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:26.970710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf754d-bd36-4998-ad9f-27d613c73be2 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:26.976185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f062e7-fe74-4bfa-b611-b0e3977d4519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:26.977063 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f062e7-fe74-4bfa-b611-b0e3977d4519 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:26.977452 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f062e7-fe74-4bfa-b611-b0e3977d4519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:26.977830 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f062e7-fe74-4bfa-b611-b0e3977d4519 admin admin] PATH_INFO: `/v3/projects/bde03995f7744cd9b6493b5f4fbbe587/users/664503abad3148b3acb9a51af502f8d2/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:26.980200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f062e7-fe74-4bfa-b611-b0e3977d4519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde03995f7744cd9b6493b5f4fbbe587, user_id=664503abad3148b3acb9a51af502f8d2, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:26.983398 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f062e7-fe74-4bfa-b611-b0e3977d4519 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:26.995954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1013/2035] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:26 2022] PUT /identity/v3/projects/bde03995f7744cd9b6493b5f4fbbe587/users/664503abad3148b3acb9a51af502f8d2/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:27.012972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a0aab-6a8e-432a-872f-139e02573782 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:27.013362 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a0aab-6a8e-432a-872f-139e02573782 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:27.013758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a0aab-6a8e-432a-872f-139e02573782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:27.050517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-940a0aab-6a8e-432a-872f-139e02573782 None None] MFA Rules not processed for user `664503abad3148b3acb9a51af502f8d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:27.051864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-940a0aab-6a8e-432a-872f-139e02573782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:27.099943 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:27.149654 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1023/2036] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:27.158985 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f062e7-fe74-4bfa-b611-b0e3977d4519 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:27.161639 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-72f062e7-fe74-4bfa-b611-b0e3977d4519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:27.251232 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f062e7-fe74-4bfa-b611-b0e3977d4519 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:27.255901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc7ed24-f2ae-43ed-afac-cc4b8730bc8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:27.257851 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7ed24-f2ae-43ed-afac-cc4b8730bc8b service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:27.257851 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7ed24-f2ae-43ed-afac-cc4b8730bc8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:27.257851 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc7ed24-f2ae-43ed-afac-cc4b8730bc8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:27.268157 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7ed24-f2ae-43ed-afac-cc4b8730bc8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:27.269731 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc7ed24-f2ae-43ed-afac-cc4b8730bc8b service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:27.281221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1014/2037] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:27.376236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a0aab-6a8e-432a-872f-139e02573782 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:27.387692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a0aab-6a8e-432a-872f-139e02573782 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:27.391522 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e204c5a-118f-4017-b794-84fe2abeffa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:27.392239 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e204c5a-118f-4017-b794-84fe2abeffa5 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:27.392592 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e204c5a-118f-4017-b794-84fe2abeffa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:27.392962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e204c5a-118f-4017-b794-84fe2abeffa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:27.488234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e204c5a-118f-4017-b794-84fe2abeffa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:27.490419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e204c5a-118f-4017-b794-84fe2abeffa5 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:27.504347 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1024/2038] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:27 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:18:27.929649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3ee4e-3ccb-4566-a4ea-6fd559a01a44 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:27.929936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3ee4e-3ccb-4566-a4ea-6fd559a01a44 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:27.930213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3ee4e-3ccb-4566-a4ea-6fd559a01a44 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:27.931612 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1015/2039] 10.222.0.45 () {54 vars in 921 bytes} [Thu Aug 4 02:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:18:27.966686 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fe418-2d5f-4c4e-bc91-bccf0f3af8a0 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:27.967034 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fe418-2d5f-4c4e-bc91-bccf0f3af8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:27.967266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fe418-2d5f-4c4e-bc91-bccf0f3af8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:27.990721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-8d1fe418-2d5f-4c4e-bc91-bccf0f3af8a0 None None] MFA Rules not processed for user `796772cf491047c8b0a609dae9402307`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:27.991985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8d1fe418-2d5f-4c4e-bc91-bccf0f3af8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:28.029014 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3ee4e-3ccb-4566-a4ea-6fd559a01a44 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:28.030918 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-17e3ee4e-3ccb-4566-a4ea-6fd559a01a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:28.072071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1025/2040] 10.222.0.45 () {58 vars in 1018 bytes} [Thu Aug 4 02:18:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:28.082814 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fe418-2d5f-4c4e-bc91-bccf0f3af8a0 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:28.084430 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8d1fe418-2d5f-4c4e-bc91-bccf0f3af8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:28.097051 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fe418-2d5f-4c4e-bc91-bccf0f3af8a0 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:28.102663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d880f57-a473-4af3-97c3-1525d321a4a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796772cf491047c8b0a609dae9402307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:28.103376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d880f57-a473-4af3-97c3-1525d321a4a1 service placement] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:28.103727 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d880f57-a473-4af3-97c3-1525d321a4a1 service placement] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:28.104004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d880f57-a473-4af3-97c3-1525d321a4a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:28.112712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d880f57-a473-4af3-97c3-1525d321a4a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:28.114263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3ee4e-3ccb-4566-a4ea-6fd559a01a44 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:28.114379 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d880f57-a473-4af3-97c3-1525d321a4a1 service placement] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:28.119301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44945979-476f-47f5-9af4-4f13b73e930a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:28.119945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44945979-476f-47f5-9af4-4f13b73e930a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:28.120254 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44945979-476f-47f5-9af4-4f13b73e930a admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:28.120531 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44945979-476f-47f5-9af4-4f13b73e930a admin admin] PATH_INFO: `/v3/users/7c4eaad01b9040fbb56c346ee7400396` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:28.121750 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44945979-476f-47f5-9af4-4f13b73e930a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c4eaad01b9040fbb56c346ee7400396)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:28.123653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44945979-476f-47f5-9af4-4f13b73e930a admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:28.124532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1026/2041] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:18:28.151857 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:28.180404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-44945979-476f-47f5-9af4-4f13b73e930a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c4eaad01b9040fbb56c346ee7400396'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:28.181092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-44945979-476f-47f5-9af4-4f13b73e930a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c4eaad01b9040fbb56c346ee7400396'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:28.181720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-44945979-476f-47f5-9af4-4f13b73e930a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c4eaad01b9040fbb56c346ee7400396'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:28.193470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-44945979-476f-47f5-9af4-4f13b73e930a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c4eaad01b9040fbb56c346ee7400396'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:28.201446 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-44945979-476f-47f5-9af4-4f13b73e930a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c4eaad01b9040fbb56c346ee7400396'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:28.215626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1016/2042] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:18:28 2022] DELETE /identity/v3/users/7c4eaad01b9040fbb56c346ee7400396 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:18:28.392201 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d880f57-a473-4af3-97c3-1525d321a4a1 service placement] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:28.405192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d880f57-a473-4af3-97c3-1525d321a4a1 service placement] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:28.458047 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:28.469275 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162d009-a636-4dbd-aa5c-3163d98ca6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:28.469933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162d009-a636-4dbd-aa5c-3163d98ca6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:28.470214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162d009-a636-4dbd-aa5c-3163d98ca6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:28.470489 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2162d009-a636-4dbd-aa5c-3163d98ca6fd admin admin] PATH_INFO: `/v3/projects/69bc670511d847b980a77302686d3717` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:28.471967 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162d009-a636-4dbd-aa5c-3163d98ca6fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=69bc670511d847b980a77302686d3717)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:28.474968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2162d009-a636-4dbd-aa5c-3163d98ca6fd admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:28.522753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2162d009-a636-4dbd-aa5c-3163d98ca6fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69bc670511d847b980a77302686d3717', 'request_id': 'req-2162d009-a636-4dbd-aa5c-3163d98ca6fd'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:28.531256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2162d009-a636-4dbd-aa5c-3163d98ca6fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69bc670511d847b980a77302686d3717', 'request_id': 'req-2162d009-a636-4dbd-aa5c-3163d98ca6fd'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:28.537010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2162d009-a636-4dbd-aa5c-3163d98ca6fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69bc670511d847b980a77302686d3717', 'request_id': 'req-2162d009-a636-4dbd-aa5c-3163d98ca6fd'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:28.548408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2162d009-a636-4dbd-aa5c-3163d98ca6fd admin admin] The token cache is being invalidate because project 69bc670511d847b980a77302686d3717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:28.548729 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2162d009-a636-4dbd-aa5c-3163d98ca6fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:28.549386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2162d009-a636-4dbd-aa5c-3163d98ca6fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:28.550901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1027/2043] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:28 2022] DELETE /identity/v3/projects/69bc670511d847b980a77302686d3717 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:28.555455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44945979-476f-47f5-9af4-4f13b73e930a admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:28.556869 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-44945979-476f-47f5-9af4-4f13b73e930a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:28.571197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44945979-476f-47f5-9af4-4f13b73e930a admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:28.628931 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44237d1e-378c-4b9f-9fa4-f7972c5e2049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:28.629588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44237d1e-378c-4b9f-9fa4-f7972c5e2049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:28.629870 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44237d1e-378c-4b9f-9fa4-f7972c5e2049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:28.630145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44237d1e-378c-4b9f-9fa4-f7972c5e2049 admin admin] PATH_INFO: `/v3/users/e506d3a50a394895b5ae54b1455b797a` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:28.631317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44237d1e-378c-4b9f-9fa4-f7972c5e2049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e506d3a50a394895b5ae54b1455b797a)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:28.633780 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44237d1e-378c-4b9f-9fa4-f7972c5e2049 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:28.682948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-44237d1e-378c-4b9f-9fa4-f7972c5e2049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e506d3a50a394895b5ae54b1455b797a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:28.683564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-44237d1e-378c-4b9f-9fa4-f7972c5e2049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e506d3a50a394895b5ae54b1455b797a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:28.684162 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-44237d1e-378c-4b9f-9fa4-f7972c5e2049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e506d3a50a394895b5ae54b1455b797a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:28.697475 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-44237d1e-378c-4b9f-9fa4-f7972c5e2049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e506d3a50a394895b5ae54b1455b797a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:28.704660 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-44237d1e-378c-4b9f-9fa4-f7972c5e2049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e506d3a50a394895b5ae54b1455b797a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:28.724535 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1017/2044] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:18:28 2022] DELETE /identity/v3/users/e506d3a50a394895b5ae54b1455b797a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:28.911430 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162d009-a636-4dbd-aa5c-3163d98ca6fd admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:28.913713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-2162d009-a636-4dbd-aa5c-3163d98ca6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:28.928224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2162d009-a636-4dbd-aa5c-3163d98ca6fd admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:28.998685 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff9cee-321f-4b3b-851e-184f0df915ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:28.999406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff9cee-321f-4b3b-851e-184f0df915ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:28.999701 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff9cee-321f-4b3b-851e-184f0df915ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:28.999973 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff9cee-321f-4b3b-851e-184f0df915ba admin admin] PATH_INFO: `/v3/projects/639e766fba9d4f358d8afb03e939e065` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:29.001608 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff9cee-321f-4b3b-851e-184f0df915ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=639e766fba9d4f358d8afb03e939e065)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:29.003414 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff9cee-321f-4b3b-851e-184f0df915ba admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:29.074846 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-f0ff9cee-321f-4b3b-851e-184f0df915ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '639e766fba9d4f358d8afb03e939e065', 'request_id': 'req-f0ff9cee-321f-4b3b-851e-184f0df915ba'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:29.084659 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-f0ff9cee-321f-4b3b-851e-184f0df915ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '639e766fba9d4f358d8afb03e939e065', 'request_id': 'req-f0ff9cee-321f-4b3b-851e-184f0df915ba'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:29.092643 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-f0ff9cee-321f-4b3b-851e-184f0df915ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '639e766fba9d4f358d8afb03e939e065', 'request_id': 'req-f0ff9cee-321f-4b3b-851e-184f0df915ba'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:29.117377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-f0ff9cee-321f-4b3b-851e-184f0df915ba admin admin] The token cache is being invalidate because project 639e766fba9d4f358d8afb03e939e065 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:29.117377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-f0ff9cee-321f-4b3b-851e-184f0df915ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:29.117377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-f0ff9cee-321f-4b3b-851e-184f0df915ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:29.117377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1028/2045] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:28 2022] DELETE /identity/v3/projects/639e766fba9d4f358d8afb03e939e065 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:18:29.121275 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db52bd-1fa9-4c41-b417-738038765339 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:29.122328 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db52bd-1fa9-4c41-b417-738038765339 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:29.122803 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db52bd-1fa9-4c41-b417-738038765339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:29.146206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-38db52bd-1fa9-4c41-b417-738038765339 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:29.147438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-38db52bd-1fa9-4c41-b417-738038765339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:29.162141 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:29.214904 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1018/2046] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:29.222573 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481fbc3-c5c2-445b-9e03-defe57e06e40 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:29.222758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481fbc3-c5c2-445b-9e03-defe57e06e40 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:29.223041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481fbc3-c5c2-445b-9e03-defe57e06e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:29.244346 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-6481fbc3-c5c2-445b-9e03-defe57e06e40 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:29.245587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-6481fbc3-c5c2-445b-9e03-defe57e06e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:29.254099 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1029/2047] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:29.258420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db52bd-1fa9-4c41-b417-738038765339 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:29.268294 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db52bd-1fa9-4c41-b417-738038765339 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:29.272979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89499d1-5c7b-48b6-aff1-5b63b864d62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:29.273576 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89499d1-5c7b-48b6-aff1-5b63b864d62d admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:29.273856 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89499d1-5c7b-48b6-aff1-5b63b864d62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:29.274128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89499d1-5c7b-48b6-aff1-5b63b864d62d admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:29.274821 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89499d1-5c7b-48b6-aff1-5b63b864d62d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:18:29.275134 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89499d1-5c7b-48b6-aff1-5b63b864d62d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:29.276761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89499d1-5c7b-48b6-aff1-5b63b864d62d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:29.289871 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1019/2048] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:18:29.294348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481fbc3-c5c2-445b-9e03-defe57e06e40 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:29.303731 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481fbc3-c5c2-445b-9e03-defe57e06e40 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:29.309119 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b9695-564d-4af1-984b-98081edf95b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:29.309810 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b9695-564d-4af1-984b-98081edf95b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:29.310290 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b9695-564d-4af1-984b-98081edf95b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:29.310768 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b9695-564d-4af1-984b-98081edf95b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:29.311795 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b9695-564d-4af1-984b-98081edf95b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:29.313716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b9695-564d-4af1-984b-98081edf95b7 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:29.319238 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:29.345564 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1030/2049] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:18:29 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:29.347936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89499d1-5c7b-48b6-aff1-5b63b864d62d admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:29.358703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89499d1-5c7b-48b6-aff1-5b63b864d62d admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:29.417666 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4748ea-3a6e-40d2-b228-620f984d5fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:29.418292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4748ea-3a6e-40d2-b228-620f984d5fac admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:29.418567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4748ea-3a6e-40d2-b228-620f984d5fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:29.418837 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4748ea-3a6e-40d2-b228-620f984d5fac admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:29.419515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4748ea-3a6e-40d2-b228-620f984d5fac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:29.421568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4748ea-3a6e-40d2-b228-620f984d5fac admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:29.446392 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1020/2050] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:18:29 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:29.447631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b9695-564d-4af1-984b-98081edf95b7 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:29.458420 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b9695-564d-4af1-984b-98081edf95b7 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:29.463040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011483b-c69d-4358-92ba-13535f72b532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:29.463655 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011483b-c69d-4358-92ba-13535f72b532 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:29.463929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011483b-c69d-4358-92ba-13535f72b532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:29.464199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011483b-c69d-4358-92ba-13535f72b532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:29.464924 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011483b-c69d-4358-92ba-13535f72b532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:29.467075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011483b-c69d-4358-92ba-13535f72b532 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:29.477793 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1031/2051] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:29.481796 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4748ea-3a6e-40d2-b228-620f984d5fac admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:29.492593 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4748ea-3a6e-40d2-b228-620f984d5fac admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:29.497398 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3614e7e-8fcb-45c5-9b41-186911582c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:29.498011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3614e7e-8fcb-45c5-9b41-186911582c92 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:29.498290 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3614e7e-8fcb-45c5-9b41-186911582c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:29.498564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3614e7e-8fcb-45c5-9b41-186911582c92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:29.499260 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3614e7e-8fcb-45c5-9b41-186911582c92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:29.500986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3614e7e-8fcb-45c5-9b41-186911582c92 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:29.511509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1021/2052] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:29.515906 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011483b-c69d-4358-92ba-13535f72b532 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:29.527054 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011483b-c69d-4358-92ba-13535f72b532 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:29.531784 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf95d6c-b896-4f7c-be73-8ddc92fd1ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:29.532445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf95d6c-b896-4f7c-be73-8ddc92fd1ea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:29.532758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf95d6c-b896-4f7c-be73-8ddc92fd1ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:29.533126 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf95d6c-b896-4f7c-be73-8ddc92fd1ea7 admin admin] PATH_INFO: `/v3/projects/b3943b3bb3194c139922c2dcc8440d13/users/3b2ba6cf026e45d39f1e692b41e6783e/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:29.552520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf95d6c-b896-4f7c-be73-8ddc92fd1ea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3943b3bb3194c139922c2dcc8440d13, user_id=3b2ba6cf026e45d39f1e692b41e6783e, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:29.555680 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf95d6c-b896-4f7c-be73-8ddc92fd1ea7 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:29.569529 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1032/2053] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:29 2022] PUT /identity/v3/projects/b3943b3bb3194c139922c2dcc8440d13/users/3b2ba6cf026e45d39f1e692b41e6783e/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:18:29.579872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0500ae3-c12a-4a5e-b5d8-d269c4b9496e None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:29.580157 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0500ae3-c12a-4a5e-b5d8-d269c4b9496e None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:29.580434 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0500ae3-c12a-4a5e-b5d8-d269c4b9496e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:29.610048 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-e0500ae3-c12a-4a5e-b5d8-d269c4b9496e None None] MFA Rules not processed for user `3b2ba6cf026e45d39f1e692b41e6783e`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:29.611449 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e0500ae3-c12a-4a5e-b5d8-d269c4b9496e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:29.687461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1022/2054] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:29.698600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf95d6c-b896-4f7c-be73-8ddc92fd1ea7 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:29.700295 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-6bf95d6c-b896-4f7c-be73-8ddc92fd1ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:29.761289 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf95d6c-b896-4f7c-be73-8ddc92fd1ea7 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:29.766129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc56c52-2337-4149-af3c-95a2f7b12568 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:29.766836 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc56c52-2337-4149-af3c-95a2f7b12568 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:29.767116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc56c52-2337-4149-af3c-95a2f7b12568 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:29.767392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc56c52-2337-4149-af3c-95a2f7b12568 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:29.786720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc56c52-2337-4149-af3c-95a2f7b12568 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:29.788197 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc56c52-2337-4149-af3c-95a2f7b12568 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:29.798545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1033/2055] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:18:29.820316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0500ae3-c12a-4a5e-b5d8-d269c4b9496e None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:29.838947 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:29.888690 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0500ae3-c12a-4a5e-b5d8-d269c4b9496e None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:29.893187 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99166569-756d-42fa-9450-cc486f5a5d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:29.893842 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99166569-756d-42fa-9450-cc486f5a5d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:29.894118 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99166569-756d-42fa-9450-cc486f5a5d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:29.894412 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99166569-756d-42fa-9450-cc486f5a5d29 admin admin] PATH_INFO: `/v3/users/3b2ba6cf026e45d39f1e692b41e6783e` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:29.895606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99166569-756d-42fa-9450-cc486f5a5d29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b2ba6cf026e45d39f1e692b41e6783e)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:29.897562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99166569-756d-42fa-9450-cc486f5a5d29 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:29.945444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-99166569-756d-42fa-9450-cc486f5a5d29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b2ba6cf026e45d39f1e692b41e6783e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:29.946084 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-99166569-756d-42fa-9450-cc486f5a5d29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b2ba6cf026e45d39f1e692b41e6783e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:29.946684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-99166569-756d-42fa-9450-cc486f5a5d29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b2ba6cf026e45d39f1e692b41e6783e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:29.958755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-99166569-756d-42fa-9450-cc486f5a5d29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b2ba6cf026e45d39f1e692b41e6783e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:29.969814 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-99166569-756d-42fa-9450-cc486f5a5d29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b2ba6cf026e45d39f1e692b41e6783e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:29.984609 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1023/2056] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:18:29 2022] DELETE /identity/v3/users/3b2ba6cf026e45d39f1e692b41e6783e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:29.989078 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7361f72-3d35-4c46-8041-85a2421a05b1 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:29.989386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7361f72-3d35-4c46-8041-85a2421a05b1 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:29.989655 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7361f72-3d35-4c46-8041-85a2421a05b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:30.009072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-b7361f72-3d35-4c46-8041-85a2421a05b1 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:30.010310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b7361f72-3d35-4c46-8041-85a2421a05b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:30.074184 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1034/2057] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:30.086317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99166569-756d-42fa-9450-cc486f5a5d29 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:30.087770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-99166569-756d-42fa-9450-cc486f5a5d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:30.157461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99166569-756d-42fa-9450-cc486f5a5d29 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:30.161441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf8956-5f39-495b-8a67-d1948f49e5c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:30.162077 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf8956-5f39-495b-8a67-d1948f49e5c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:30.162462 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf8956-5f39-495b-8a67-d1948f49e5c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:30.162637 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf8956-5f39-495b-8a67-d1948f49e5c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:30.172378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf8956-5f39-495b-8a67-d1948f49e5c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:30.173790 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf8956-5f39-495b-8a67-d1948f49e5c4 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:30.183711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1024/2058] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:30.528715 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7361f72-3d35-4c46-8041-85a2421a05b1 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:30.530699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b7361f72-3d35-4c46-8041-85a2421a05b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:30.543526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7361f72-3d35-4c46-8041-85a2421a05b1 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:30.549079 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fd177-2b94-42cf-997b-58cef958cd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:30.550066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fd177-2b94-42cf-997b-58cef958cd59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:30.550568 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fd177-2b94-42cf-997b-58cef958cd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:30.550967 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fd177-2b94-42cf-997b-58cef958cd59 admin admin] PATH_INFO: `/v3/projects/b3943b3bb3194c139922c2dcc8440d13` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:30.552792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fd177-2b94-42cf-997b-58cef958cd59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3943b3bb3194c139922c2dcc8440d13)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:30.555488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fd177-2b94-42cf-997b-58cef958cd59 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:30.602648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6a4fd177-2b94-42cf-997b-58cef958cd59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3943b3bb3194c139922c2dcc8440d13', 'request_id': 'req-6a4fd177-2b94-42cf-997b-58cef958cd59'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:30.604325 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:30.610975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6a4fd177-2b94-42cf-997b-58cef958cd59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3943b3bb3194c139922c2dcc8440d13', 'request_id': 'req-6a4fd177-2b94-42cf-997b-58cef958cd59'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:30.618242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6a4fd177-2b94-42cf-997b-58cef958cd59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3943b3bb3194c139922c2dcc8440d13', 'request_id': 'req-6a4fd177-2b94-42cf-997b-58cef958cd59'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:30.630258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6a4fd177-2b94-42cf-997b-58cef958cd59 admin admin] The token cache is being invalidate because project b3943b3bb3194c139922c2dcc8440d13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:30.630634 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6a4fd177-2b94-42cf-997b-58cef958cd59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:30.631441 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6a4fd177-2b94-42cf-997b-58cef958cd59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:30.633957 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1035/2059] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:30 2022] DELETE /identity/v3/projects/b3943b3bb3194c139922c2dcc8440d13 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:30.638926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81c6d6-aa3f-4a83-af8a-4c0b8271d6f5 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:30.639206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81c6d6-aa3f-4a83-af8a-4c0b8271d6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:30.639478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81c6d6-aa3f-4a83-af8a-4c0b8271d6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:30.660290 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-4b81c6d6-aa3f-4a83-af8a-4c0b8271d6f5 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:30.661400 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-4b81c6d6-aa3f-4a83-af8a-4c0b8271d6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:30.696509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:30.737178 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1025/2060] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:30.745374 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2bede-ce25-41bb-9eed-bd94e4612e52 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:30.745662 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2bede-ce25-41bb-9eed-bd94e4612e52 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:30.745944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2bede-ce25-41bb-9eed-bd94e4612e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:30.768356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-29c2bede-ce25-41bb-9eed-bd94e4612e52 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:30.769614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-29c2bede-ce25-41bb-9eed-bd94e4612e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:30.777607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1036/2061] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:30.781958 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81c6d6-aa3f-4a83-af8a-4c0b8271d6f5 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:30.792105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81c6d6-aa3f-4a83-af8a-4c0b8271d6f5 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:30.796703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2774b6b-60aa-4ca5-8c46-3c129d4b46a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:30.797339 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2774b6b-60aa-4ca5-8c46-3c129d4b46a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:30.797620 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2774b6b-60aa-4ca5-8c46-3c129d4b46a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:30.797896 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2774b6b-60aa-4ca5-8c46-3c129d4b46a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:30.798596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2774b6b-60aa-4ca5-8c46-3c129d4b46a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:18:30.798902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2774b6b-60aa-4ca5-8c46-3c129d4b46a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:30.800498 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2774b6b-60aa-4ca5-8c46-3c129d4b46a0 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:30.812796 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1026/2062] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:18:30.817352 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2bede-ce25-41bb-9eed-bd94e4612e52 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:30.827035 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2bede-ce25-41bb-9eed-bd94e4612e52 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:30.831877 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b740e-ab01-4d0d-a53f-f0f4042c8858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:30.832498 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b740e-ab01-4d0d-a53f-f0f4042c8858 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:30.832840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b740e-ab01-4d0d-a53f-f0f4042c8858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:30.833120 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b740e-ab01-4d0d-a53f-f0f4042c8858 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:30.834038 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b740e-ab01-4d0d-a53f-f0f4042c8858 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:30.835997 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b740e-ab01-4d0d-a53f-f0f4042c8858 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:30.862944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1037/2063] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:18:30 2022] POST /identity/v3/projects => generated 361 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:18:30.867786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2774b6b-60aa-4ca5-8c46-3c129d4b46a0 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:30.877807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2774b6b-60aa-4ca5-8c46-3c129d4b46a0 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:30.938284 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53f419-3266-4cc7-821b-ce5bdbd119de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:30.938810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53f419-3266-4cc7-821b-ce5bdbd119de admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:30.939139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53f419-3266-4cc7-821b-ce5bdbd119de admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:30.939686 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53f419-3266-4cc7-821b-ce5bdbd119de admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:30.940075 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53f419-3266-4cc7-821b-ce5bdbd119de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:30.942034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d53f419-3266-4cc7-821b-ce5bdbd119de admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:30.962898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1027/2064] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:18:30 2022] POST /identity/v3/users => generated 281 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:30.967154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b740e-ab01-4d0d-a53f-f0f4042c8858 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:30.979126 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b740e-ab01-4d0d-a53f-f0f4042c8858 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:30.983847 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8eeef-f8af-4911-af34-c80ee9a39653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:30.984467 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8eeef-f8af-4911-af34-c80ee9a39653 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:30.984780 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8eeef-f8af-4911-af34-c80ee9a39653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:30.985053 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d8eeef-f8af-4911-af34-c80ee9a39653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:30.985749 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8eeef-f8af-4911-af34-c80ee9a39653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:30.987354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d8eeef-f8af-4911-af34-c80ee9a39653 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:30.998427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1038/2065] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:18:31.002916 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53f419-3266-4cc7-821b-ce5bdbd119de admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:31.013189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d53f419-3266-4cc7-821b-ce5bdbd119de admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:31.017775 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e7b0c1-8d18-47fc-be13-0d858eba9f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:31.018380 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7b0c1-8d18-47fc-be13-0d858eba9f2c admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:31.018663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7b0c1-8d18-47fc-be13-0d858eba9f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:31.018924 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e7b0c1-8d18-47fc-be13-0d858eba9f2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:31.019613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7b0c1-8d18-47fc-be13-0d858eba9f2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:31.021325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e7b0c1-8d18-47fc-be13-0d858eba9f2c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:31.032832 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1028/2066] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:31.036804 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8eeef-f8af-4911-af34-c80ee9a39653 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:31.047545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d8eeef-f8af-4911-af34-c80ee9a39653 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:31.052448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00184655-28a8-467e-9e36-19ef660282df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:31.053157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00184655-28a8-467e-9e36-19ef660282df admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:31.053439 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00184655-28a8-467e-9e36-19ef660282df admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:31.053712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00184655-28a8-467e-9e36-19ef660282df admin admin] PATH_INFO: `/v3/projects/565ae7cf31ac42b390f4e64d9accd02c/users/5a46b3be8eb14e1394db1a633f295203/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:31.069195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00184655-28a8-467e-9e36-19ef660282df admin admin] RBAC: Authorizing `identity:create_grant(project_id=565ae7cf31ac42b390f4e64d9accd02c, user_id=5a46b3be8eb14e1394db1a633f295203, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:31.071916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00184655-28a8-467e-9e36-19ef660282df admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:31.086419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1039/2067] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:31 2022] PUT /identity/v3/projects/565ae7cf31ac42b390f4e64d9accd02c/users/5a46b3be8eb14e1394db1a633f295203/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:31.096997 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7f2e5-b22e-42a6-bc48-731eb89c9f7e None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:31.097278 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7f2e5-b22e-42a6-bc48-731eb89c9f7e None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:31.097546 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef7f2e5-b22e-42a6-bc48-731eb89c9f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:31.126220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-2ef7f2e5-b22e-42a6-bc48-731eb89c9f7e None None] MFA Rules not processed for user `5a46b3be8eb14e1394db1a633f295203`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:31.127431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2ef7f2e5-b22e-42a6-bc48-731eb89c9f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:31.195692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1029/2068] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:31.229927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00184655-28a8-467e-9e36-19ef660282df admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:31.231939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-00184655-28a8-467e-9e36-19ef660282df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:31.290388 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00184655-28a8-467e-9e36-19ef660282df admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:31.294197 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d24239-45f9-4f8b-890a-d6d7d6a3933d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:31.294821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d24239-45f9-4f8b-890a-d6d7d6a3933d service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:31.295104 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d24239-45f9-4f8b-890a-d6d7d6a3933d service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:31.295383 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d24239-45f9-4f8b-890a-d6d7d6a3933d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:31.305760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d24239-45f9-4f8b-890a-d6d7d6a3933d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:31.307200 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d24239-45f9-4f8b-890a-d6d7d6a3933d service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:31.316518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1040/2069] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:31.332528 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7f2e5-b22e-42a6-bc48-731eb89c9f7e None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:31.371060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:31.418512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef7f2e5-b22e-42a6-bc48-731eb89c9f7e None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:31.425441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab446965-1c24-4ff7-b617-7cfa5159431c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:31.426361 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab446965-1c24-4ff7-b617-7cfa5159431c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:31.426824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab446965-1c24-4ff7-b617-7cfa5159431c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:31.427248 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab446965-1c24-4ff7-b617-7cfa5159431c admin admin] PATH_INFO: `/v3/users/5a46b3be8eb14e1394db1a633f295203` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:31.428993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab446965-1c24-4ff7-b617-7cfa5159431c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a46b3be8eb14e1394db1a633f295203)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:31.431306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab446965-1c24-4ff7-b617-7cfa5159431c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:31.505239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ab446965-1c24-4ff7-b617-7cfa5159431c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a46b3be8eb14e1394db1a633f295203'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:31.506131 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ab446965-1c24-4ff7-b617-7cfa5159431c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a46b3be8eb14e1394db1a633f295203'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:31.506904 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ab446965-1c24-4ff7-b617-7cfa5159431c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a46b3be8eb14e1394db1a633f295203'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:31.528314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ab446965-1c24-4ff7-b617-7cfa5159431c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a46b3be8eb14e1394db1a633f295203'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:31.540807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ab446965-1c24-4ff7-b617-7cfa5159431c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a46b3be8eb14e1394db1a633f295203'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:31.569368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1030/2070] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:18:31 2022] DELETE /identity/v3/users/5a46b3be8eb14e1394db1a633f295203 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:31.574177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e6858-4b47-4514-948e-0307671c18f5 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:31.574677 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e6858-4b47-4514-948e-0307671c18f5 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:31.574946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e6858-4b47-4514-948e-0307671c18f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:31.617040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-085e6858-4b47-4514-948e-0307671c18f5 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:31.618507 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-085e6858-4b47-4514-948e-0307671c18f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:31.691875 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1041/2071] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:31.703912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab446965-1c24-4ff7-b617-7cfa5159431c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:31.705357 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ab446965-1c24-4ff7-b617-7cfa5159431c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:31.780881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab446965-1c24-4ff7-b617-7cfa5159431c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:31.784221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c32b6-4ec4-4f44-a77a-552745c598ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:31.785058 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c32b6-4ec4-4f44-a77a-552745c598ff service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:31.785520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c32b6-4ec4-4f44-a77a-552745c598ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:31.785892 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c32b6-4ec4-4f44-a77a-552745c598ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:31.797628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c32b6-4ec4-4f44-a77a-552745c598ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:31.799199 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c32b6-4ec4-4f44-a77a-552745c598ff service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:31.809970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1031/2072] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:32.077502 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e6858-4b47-4514-948e-0307671c18f5 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:32.079755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-085e6858-4b47-4514-948e-0307671c18f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:32.091337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e6858-4b47-4514-948e-0307671c18f5 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:32.096149 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ef026-73d8-4fe5-93c2-dd1b929120de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:32.096826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ef026-73d8-4fe5-93c2-dd1b929120de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:32.097169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ef026-73d8-4fe5-93c2-dd1b929120de admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:32.097451 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ef026-73d8-4fe5-93c2-dd1b929120de admin admin] PATH_INFO: `/v3/projects/565ae7cf31ac42b390f4e64d9accd02c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:32.098621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ef026-73d8-4fe5-93c2-dd1b929120de admin admin] RBAC: Authorizing `identity:delete_project(project_id=565ae7cf31ac42b390f4e64d9accd02c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:32.100525 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ef026-73d8-4fe5-93c2-dd1b929120de admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:32.113622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=94 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:32.152586 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-540ef026-73d8-4fe5-93c2-dd1b929120de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '565ae7cf31ac42b390f4e64d9accd02c', 'request_id': 'req-540ef026-73d8-4fe5-93c2-dd1b929120de'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:32.160730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-540ef026-73d8-4fe5-93c2-dd1b929120de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '565ae7cf31ac42b390f4e64d9accd02c', 'request_id': 'req-540ef026-73d8-4fe5-93c2-dd1b929120de'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:32.166963 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-540ef026-73d8-4fe5-93c2-dd1b929120de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '565ae7cf31ac42b390f4e64d9accd02c', 'request_id': 'req-540ef026-73d8-4fe5-93c2-dd1b929120de'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:32.178600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-540ef026-73d8-4fe5-93c2-dd1b929120de admin admin] The token cache is being invalidate because project 565ae7cf31ac42b390f4e64d9accd02c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:32.178892 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-540ef026-73d8-4fe5-93c2-dd1b929120de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:32.180270 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-540ef026-73d8-4fe5-93c2-dd1b929120de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:32.181852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1042/2073] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:32 2022] DELETE /identity/v3/projects/565ae7cf31ac42b390f4e64d9accd02c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:32.186980 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7eea5-9cf4-4827-8638-bf6cd65bd0fb None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:32.187266 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7eea5-9cf4-4827-8638-bf6cd65bd0fb None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:32.187564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7eea5-9cf4-4827-8638-bf6cd65bd0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:32.207661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-a7a7eea5-9cf4-4827-8638-bf6cd65bd0fb None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:32.209077 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a7a7eea5-9cf4-4827-8638-bf6cd65bd0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:32.259573 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:32.273544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1032/2074] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:32.281600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1510a-979b-4df4-a258-e7bb79e00922 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:32.281886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1510a-979b-4df4-a258-e7bb79e00922 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:32.282161 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d1510a-979b-4df4-a258-e7bb79e00922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:32.305049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-a2d1510a-979b-4df4-a258-e7bb79e00922 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:32.306262 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a2d1510a-979b-4df4-a258-e7bb79e00922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:32.314152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1043/2075] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:32.320042 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7eea5-9cf4-4827-8638-bf6cd65bd0fb None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:32.329999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7eea5-9cf4-4827-8638-bf6cd65bd0fb None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:32.334898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea33cfe-143a-4eb2-9250-372b3b8ae95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:32.335514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea33cfe-143a-4eb2-9250-372b3b8ae95a admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:32.335794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea33cfe-143a-4eb2-9250-372b3b8ae95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:32.336071 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea33cfe-143a-4eb2-9250-372b3b8ae95a admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:32.336812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea33cfe-143a-4eb2-9250-372b3b8ae95a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:18:32.337118 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea33cfe-143a-4eb2-9250-372b3b8ae95a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:32.338754 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea33cfe-143a-4eb2-9250-372b3b8ae95a admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:32.350628 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1033/2076] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:18:32.356093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1510a-979b-4df4-a258-e7bb79e00922 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:32.365878 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d1510a-979b-4df4-a258-e7bb79e00922 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:32.371653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e5aab-c1bd-4339-9543-f21c475ed483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:32.372280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e5aab-c1bd-4339-9543-f21c475ed483 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:32.372566 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e5aab-c1bd-4339-9543-f21c475ed483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:32.372926 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620e5aab-c1bd-4339-9543-f21c475ed483 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:32.373596 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e5aab-c1bd-4339-9543-f21c475ed483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:32.375521 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620e5aab-c1bd-4339-9543-f21c475ed483 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:32.408454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1044/2077] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:18:32 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:18:32.410096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea33cfe-143a-4eb2-9250-372b3b8ae95a admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:32.420849 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea33cfe-143a-4eb2-9250-372b3b8ae95a admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:32.478755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ed332-1eef-4ce4-84b6-638ebfea2b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:32.479282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ed332-1eef-4ce4-84b6-638ebfea2b1f admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:32.479564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ed332-1eef-4ce4-84b6-638ebfea2b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:32.479838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251ed332-1eef-4ce4-84b6-638ebfea2b1f admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:32.480491 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ed332-1eef-4ce4-84b6-638ebfea2b1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:32.482421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251ed332-1eef-4ce4-84b6-638ebfea2b1f admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:32.512387 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1034/2078] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:18:32 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:32.516558 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e5aab-c1bd-4339-9543-f21c475ed483 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:32.527562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620e5aab-c1bd-4339-9543-f21c475ed483 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:32.532117 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f10bbf-a4e5-4cf1-a19a-3411d0cc93fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:32.532758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f10bbf-a4e5-4cf1-a19a-3411d0cc93fb admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:32.533042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f10bbf-a4e5-4cf1-a19a-3411d0cc93fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:32.533317 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f10bbf-a4e5-4cf1-a19a-3411d0cc93fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:32.534006 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f10bbf-a4e5-4cf1-a19a-3411d0cc93fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:32.535594 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f10bbf-a4e5-4cf1-a19a-3411d0cc93fb admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:32.546814 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1045/2079] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:32.551010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ed332-1eef-4ce4-84b6-638ebfea2b1f admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:32.561300 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251ed332-1eef-4ce4-84b6-638ebfea2b1f admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:32.566292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32feaf-c749-42ca-abbe-aa8659a7e4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:32.566932 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32feaf-c749-42ca-abbe-aa8659a7e4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:32.567211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32feaf-c749-42ca-abbe-aa8659a7e4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:32.567478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32feaf-c749-42ca-abbe-aa8659a7e4b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:32.568168 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32feaf-c749-42ca-abbe-aa8659a7e4b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:32.569827 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32feaf-c749-42ca-abbe-aa8659a7e4b7 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:32.580558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1035/2080] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:32.585018 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f10bbf-a4e5-4cf1-a19a-3411d0cc93fb admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:32.596854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f10bbf-a4e5-4cf1-a19a-3411d0cc93fb admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:32.602323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1e378-a73f-4a55-9d0f-7600f3763851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:32.603019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1e378-a73f-4a55-9d0f-7600f3763851 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:32.603294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1e378-a73f-4a55-9d0f-7600f3763851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:32.603565 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1e378-a73f-4a55-9d0f-7600f3763851 admin admin] PATH_INFO: `/v3/projects/48536d0e5f39411792d524f349eda2b8/users/cca6c55bfff843ab8c4c3d007e3a2ffc/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:32.618286 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1e378-a73f-4a55-9d0f-7600f3763851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48536d0e5f39411792d524f349eda2b8, user_id=cca6c55bfff843ab8c4c3d007e3a2ffc, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:32.620872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1e378-a73f-4a55-9d0f-7600f3763851 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:32.636290 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1046/2081] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:32 2022] PUT /identity/v3/projects/48536d0e5f39411792d524f349eda2b8/users/cca6c55bfff843ab8c4c3d007e3a2ffc/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:18:32.647101 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe7875-95ce-40e7-8ff4-74bff140d162 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:32.647386 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe7875-95ce-40e7-8ff4-74bff140d162 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:32.647659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe7875-95ce-40e7-8ff4-74bff140d162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:32.678112 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-abfe7875-95ce-40e7-8ff4-74bff140d162 None None] MFA Rules not processed for user `cca6c55bfff843ab8c4c3d007e3a2ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:32.679347 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-abfe7875-95ce-40e7-8ff4-74bff140d162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:32.747720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1036/2082] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:32.753130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1e378-a73f-4a55-9d0f-7600f3763851 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:32.821881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1e378-a73f-4a55-9d0f-7600f3763851 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:32.838332 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd1b48-0166-4f00-9495-4cd146cbb37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:32.839091 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd1b48-0166-4f00-9495-4cd146cbb37a admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:32.839476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd1b48-0166-4f00-9495-4cd146cbb37a admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:32.839855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd1b48-0166-4f00-9495-4cd146cbb37a admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:32.840658 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd1b48-0166-4f00-9495-4cd146cbb37a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:32.842838 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd1b48-0166-4f00-9495-4cd146cbb37a admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:32.876331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1047/2083] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:18:32 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:18:32.877562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe7875-95ce-40e7-8ff4-74bff140d162 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:32.891495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe7875-95ce-40e7-8ff4-74bff140d162 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:32.941767 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:32.955849 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784a1e8-649d-4c6e-ba66-52810a9b2c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:32.956478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784a1e8-649d-4c6e-ba66-52810a9b2c94 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:32.956794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784a1e8-649d-4c6e-ba66-52810a9b2c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:32.957067 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784a1e8-649d-4c6e-ba66-52810a9b2c94 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:32.957736 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784a1e8-649d-4c6e-ba66-52810a9b2c94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:32.960031 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784a1e8-649d-4c6e-ba66-52810a9b2c94 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:32.989041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1037/2084] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:18:32 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:18:32.993624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd1b48-0166-4f00-9495-4cd146cbb37a admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:33.005500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd1b48-0166-4f00-9495-4cd146cbb37a admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:33.010624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55683e37-e7a2-444d-895e-199f4f63181b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:33.011239 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55683e37-e7a2-444d-895e-199f4f63181b admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:33.011528 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55683e37-e7a2-444d-895e-199f4f63181b admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:33.011788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55683e37-e7a2-444d-895e-199f4f63181b admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:33.012481 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55683e37-e7a2-444d-895e-199f4f63181b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:33.014541 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55683e37-e7a2-444d-895e-199f4f63181b admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:33.027392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1048/2085] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:33.031668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784a1e8-649d-4c6e-ba66-52810a9b2c94 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:33.041835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784a1e8-649d-4c6e-ba66-52810a9b2c94 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:33.046450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642e566-7b54-453a-8b5e-9261bcba7c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:33.047050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642e566-7b54-453a-8b5e-9261bcba7c19 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:33.047341 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642e566-7b54-453a-8b5e-9261bcba7c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:33.047617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642e566-7b54-453a-8b5e-9261bcba7c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:33.048313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642e566-7b54-453a-8b5e-9261bcba7c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:33.049955 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2642e566-7b54-453a-8b5e-9261bcba7c19 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:33.064342 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1038/2086] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:33.068579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55683e37-e7a2-444d-895e-199f4f63181b admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:33.079431 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55683e37-e7a2-444d-895e-199f4f63181b admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:33.084372 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cccaa7-7577-46e7-959c-9727de691571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:33.085061 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cccaa7-7577-46e7-959c-9727de691571 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:33.085393 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cccaa7-7577-46e7-959c-9727de691571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:33.086297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cccaa7-7577-46e7-959c-9727de691571 admin admin] PATH_INFO: `/v3/projects/11bd4bf74ace4feeb93ff91cfc90a496/users/0b3f78e7c03a44e8967bb6702a9e2e2a/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:33.094509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:33.102658 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cccaa7-7577-46e7-959c-9727de691571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11bd4bf74ace4feeb93ff91cfc90a496, user_id=0b3f78e7c03a44e8967bb6702a9e2e2a, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:33.105458 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cccaa7-7577-46e7-959c-9727de691571 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:33.123776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1049/2087] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:33 2022] PUT /identity/v3/projects/11bd4bf74ace4feeb93ff91cfc90a496/users/0b3f78e7c03a44e8967bb6702a9e2e2a/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:33.134519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93782ed8-6cd2-4a66-b424-2bda862e302f None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:33.134927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93782ed8-6cd2-4a66-b424-2bda862e302f None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:33.135213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93782ed8-6cd2-4a66-b424-2bda862e302f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:33.165532 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-93782ed8-6cd2-4a66-b424-2bda862e302f None None] MFA Rules not processed for user `0b3f78e7c03a44e8967bb6702a9e2e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:33.166766 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-93782ed8-6cd2-4a66-b424-2bda862e302f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:33.232629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1039/2088] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:33.237501 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cccaa7-7577-46e7-959c-9727de691571 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:33.300985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cccaa7-7577-46e7-959c-9727de691571 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:33.306098 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78355f0-45b9-4428-90f5-5f6c3f11f8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:33.306705 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78355f0-45b9-4428-90f5-5f6c3f11f8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:33.306981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78355f0-45b9-4428-90f5-5f6c3f11f8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:33.307251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78355f0-45b9-4428-90f5-5f6c3f11f8d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:33.307907 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78355f0-45b9-4428-90f5-5f6c3f11f8d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:33.309820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78355f0-45b9-4428-90f5-5f6c3f11f8d9 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:33.337660 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1050/2089] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:18:33 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:33.341792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93782ed8-6cd2-4a66-b424-2bda862e302f None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:33.352442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93782ed8-6cd2-4a66-b424-2bda862e302f None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:33.412515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06361117-602f-433b-bfb5-1fe7d5006a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:33.412986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06361117-602f-433b-bfb5-1fe7d5006a4c admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:33.413268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06361117-602f-433b-bfb5-1fe7d5006a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:33.414105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06361117-602f-433b-bfb5-1fe7d5006a4c admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:33.414835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06361117-602f-433b-bfb5-1fe7d5006a4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:33.416835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06361117-602f-433b-bfb5-1fe7d5006a4c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:33.439959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1040/2090] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:18:33 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:33.443962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78355f0-45b9-4428-90f5-5f6c3f11f8d9 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:33.454288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78355f0-45b9-4428-90f5-5f6c3f11f8d9 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:33.459635 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac028f-556a-420f-8e41-206e77bf1992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:33.460227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac028f-556a-420f-8e41-206e77bf1992 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:33.461743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac028f-556a-420f-8e41-206e77bf1992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:33.462061 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac028f-556a-420f-8e41-206e77bf1992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:33.462807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac028f-556a-420f-8e41-206e77bf1992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:33.465056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac028f-556a-420f-8e41-206e77bf1992 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:33.480306 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1051/2091] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:33.481795 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06361117-602f-433b-bfb5-1fe7d5006a4c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:33.491954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06361117-602f-433b-bfb5-1fe7d5006a4c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:33.496597 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25924ad5-e87d-420f-8ff6-b98a59d6f042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:33.497312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25924ad5-e87d-420f-8ff6-b98a59d6f042 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:33.497579 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25924ad5-e87d-420f-8ff6-b98a59d6f042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:33.497854 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25924ad5-e87d-420f-8ff6-b98a59d6f042 admin admin] PATH_INFO: `/v3/projects/254dd7e132eb4feb80dc592337000fe8/users/ab21f5ab4d154eb2881e920660c7e588/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:33.512428 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25924ad5-e87d-420f-8ff6-b98a59d6f042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=254dd7e132eb4feb80dc592337000fe8, user_id=ab21f5ab4d154eb2881e920660c7e588, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:33.515121 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25924ad5-e87d-420f-8ff6-b98a59d6f042 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:33.528877 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1041/2092] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:33 2022] PUT /identity/v3/projects/254dd7e132eb4feb80dc592337000fe8/users/ab21f5ab4d154eb2881e920660c7e588/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:33.532989 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac028f-556a-420f-8e41-206e77bf1992 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:33.543710 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac028f-556a-420f-8e41-206e77bf1992 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:33.606944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9386357-d171-4e16-987d-8c23b6dcadab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:33.607588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9386357-d171-4e16-987d-8c23b6dcadab admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:33.607870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9386357-d171-4e16-987d-8c23b6dcadab admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:33.608143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9386357-d171-4e16-987d-8c23b6dcadab admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:33.608891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9386357-d171-4e16-987d-8c23b6dcadab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:33.610482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9386357-d171-4e16-987d-8c23b6dcadab admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:33.621363 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1052/2093] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:33.625585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25924ad5-e87d-420f-8ff6-b98a59d6f042 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:33.635905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25924ad5-e87d-420f-8ff6-b98a59d6f042 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:33.640479 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e4d76-e3eb-4643-8d6b-f70eae822ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:33.641292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e4d76-e3eb-4643-8d6b-f70eae822ebd admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:33.641455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e4d76-e3eb-4643-8d6b-f70eae822ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:33.641738 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e4d76-e3eb-4643-8d6b-f70eae822ebd admin admin] PATH_INFO: `/v3/projects/254dd7e132eb4feb80dc592337000fe8/users/ab21f5ab4d154eb2881e920660c7e588/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:33.643690 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e4d76-e3eb-4643-8d6b-f70eae822ebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=254dd7e132eb4feb80dc592337000fe8, user_id=ab21f5ab4d154eb2881e920660c7e588, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:33.646392 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e4d76-e3eb-4643-8d6b-f70eae822ebd admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:33.661318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1042/2094] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:33 2022] PUT /identity/v3/projects/254dd7e132eb4feb80dc592337000fe8/users/ab21f5ab4d154eb2881e920660c7e588/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:33.673139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48265d-ca62-4c47-89a0-cebe0be7d018 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:33.673426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48265d-ca62-4c47-89a0-cebe0be7d018 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:33.673703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c48265d-ca62-4c47-89a0-cebe0be7d018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:33.702464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-4c48265d-ca62-4c47-89a0-cebe0be7d018 None None] MFA Rules not processed for user `ab21f5ab4d154eb2881e920660c7e588`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:33.703682 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4c48265d-ca62-4c47-89a0-cebe0be7d018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:33.751159 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:33.775560 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1053/2095] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:33.787047 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e4d76-e3eb-4643-8d6b-f70eae822ebd admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:33.789367 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a73e4d76-e3eb-4643-8d6b-f70eae822ebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:33.800232 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:33.847314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e4d76-e3eb-4643-8d6b-f70eae822ebd admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:33.851476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cee10-4708-4cf8-947a-7cd0c7df7fb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:33.852141 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cee10-4708-4cf8-947a-7cd0c7df7fb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:33.852492 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cee10-4708-4cf8-947a-7cd0c7df7fb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:33.852695 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6cee10-4708-4cf8-947a-7cd0c7df7fb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:33.915269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6cee10-4708-4cf8-947a-7cd0c7df7fb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:33.917205 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6cee10-4708-4cf8-947a-7cd0c7df7fb7 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:33.926277 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1043/2096] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:38.615327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48265d-ca62-4c47-89a0-cebe0be7d018 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:38.625448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c48265d-ca62-4c47-89a0-cebe0be7d018 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:38.629509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25c555-bb61-4aff-b99b-fde19b9f59e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:38.630122 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25c555-bb61-4aff-b99b-fde19b9f59e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:38.630399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25c555-bb61-4aff-b99b-fde19b9f59e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:38.630677 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25c555-bb61-4aff-b99b-fde19b9f59e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:38.690820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25c555-bb61-4aff-b99b-fde19b9f59e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:38.692258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25c555-bb61-4aff-b99b-fde19b9f59e4 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:38.702067 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1054/2097] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:39.478693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cee10-4708-4cf8-947a-7cd0c7df7fb7 service cinder] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:39.487532 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6cee10-4708-4cf8-947a-7cd0c7df7fb7 service cinder] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:39.491437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d1510-5580-40e7-a438-cdf74a2125ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:39.492098 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d1510-5580-40e7-a438-cdf74a2125ea service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:39.492379 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d1510-5580-40e7-a438-cdf74a2125ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:39.492683 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d1510-5580-40e7-a438-cdf74a2125ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:39.502461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d1510-5580-40e7-a438-cdf74a2125ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:39.504063 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d1510-5580-40e7-a438-cdf74a2125ea service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:39.513784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1044/2098] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:18:44.117480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25c555-bb61-4aff-b99b-fde19b9f59e4 service cinder] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:44.121902 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7c25c555-bb61-4aff-b99b-fde19b9f59e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:44.214474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25c555-bb61-4aff-b99b-fde19b9f59e4 service cinder] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:44.228294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb6cde-93e0-4f83-9abc-71ebdcdfb2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:44.228294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb6cde-93e0-4f83-9abc-71ebdcdfb2de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:44.228294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb6cde-93e0-4f83-9abc-71ebdcdfb2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:44.228294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb6cde-93e0-4f83-9abc-71ebdcdfb2de admin admin] PATH_INFO: `/v3/users/a55e354fcc69463693437b07890644ea` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:44.228597 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb6cde-93e0-4f83-9abc-71ebdcdfb2de admin admin] RBAC: Authorizing `identity:delete_user(user_id=a55e354fcc69463693437b07890644ea)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:44.229981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb6cde-93e0-4f83-9abc-71ebdcdfb2de admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:44.299652 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fceb6cde-93e0-4f83-9abc-71ebdcdfb2de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a55e354fcc69463693437b07890644ea'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:44.308248 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fceb6cde-93e0-4f83-9abc-71ebdcdfb2de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a55e354fcc69463693437b07890644ea'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:44.308991 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fceb6cde-93e0-4f83-9abc-71ebdcdfb2de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a55e354fcc69463693437b07890644ea'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:44.324005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fceb6cde-93e0-4f83-9abc-71ebdcdfb2de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a55e354fcc69463693437b07890644ea'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:44.325721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fceb6cde-93e0-4f83-9abc-71ebdcdfb2de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a55e354fcc69463693437b07890644ea'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:44.337881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:44.343855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1055/2099] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:18:44 2022] DELETE /identity/v3/users/a55e354fcc69463693437b07890644ea => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:44.348212 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a2a31-309a-4280-9200-7abdfd5b961d None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:44.348774 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a2a31-309a-4280-9200-7abdfd5b961d None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:44.348884 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3a2a31-309a-4280-9200-7abdfd5b961d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:44.369609 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-9c3a2a31-309a-4280-9200-7abdfd5b961d None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:44.370814 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9c3a2a31-309a-4280-9200-7abdfd5b961d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:44.441954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1045/2100] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:44.453719 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb6cde-93e0-4f83-9abc-71ebdcdfb2de admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:44.455200 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-fceb6cde-93e0-4f83-9abc-71ebdcdfb2de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:44.533425 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb6cde-93e0-4f83-9abc-71ebdcdfb2de admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:44.537981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90d255-7fc2-4d17-a445-a408cfae55ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:44.538701 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90d255-7fc2-4d17-a445-a408cfae55ea service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:44.544916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90d255-7fc2-4d17-a445-a408cfae55ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:44.545268 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90d255-7fc2-4d17-a445-a408cfae55ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:44.557576 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90d255-7fc2-4d17-a445-a408cfae55ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:44.559079 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe90d255-7fc2-4d17-a445-a408cfae55ea service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:44.571562 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1056/2101] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:18:44.864594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a2a31-309a-4280-9200-7abdfd5b961d None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:44.866368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9c3a2a31-309a-4280-9200-7abdfd5b961d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:44.880011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3a2a31-309a-4280-9200-7abdfd5b961d None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:44.885927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:44.886520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:44.886892 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:44.887236 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf admin admin] PATH_INFO: `/v3/projects/df4b8fbecd0e4e25afe62d7d73a47f89` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:44.889457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=df4b8fbecd0e4e25afe62d7d73a47f89)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:44.891833 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:44.960883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:44.963816 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df4b8fbecd0e4e25afe62d7d73a47f89', 'request_id': 'req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:44.983059 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df4b8fbecd0e4e25afe62d7d73a47f89', 'request_id': 'req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:44.990137 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df4b8fbecd0e4e25afe62d7d73a47f89', 'request_id': 'req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:45.001728 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf admin admin] The token cache is being invalidate because project df4b8fbecd0e4e25afe62d7d73a47f89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:45.002024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:45.002699 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:45.004238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1046/2102] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:44 2022] DELETE /identity/v3/projects/df4b8fbecd0e4e25afe62d7d73a47f89 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:45.008859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90d255-7fc2-4d17-a445-a408cfae55ea service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:45.019287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90d255-7fc2-4d17-a445-a408cfae55ea service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:45.085306 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc4495-df1d-428d-ab61-a1699d8bcbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:45.085949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc4495-df1d-428d-ab61-a1699d8bcbb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:45.086233 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc4495-df1d-428d-ab61-a1699d8bcbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:45.087083 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc4495-df1d-428d-ab61-a1699d8bcbb3 admin admin] PATH_INFO: `/v3/users/664503abad3148b3acb9a51af502f8d2` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:45.088267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc4495-df1d-428d-ab61-a1699d8bcbb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=664503abad3148b3acb9a51af502f8d2)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:45.090342 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc4495-df1d-428d-ab61-a1699d8bcbb3 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:45.139831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5bbc4495-df1d-428d-ab61-a1699d8bcbb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '664503abad3148b3acb9a51af502f8d2'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:45.147105 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5bbc4495-df1d-428d-ab61-a1699d8bcbb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '664503abad3148b3acb9a51af502f8d2'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:45.147788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5bbc4495-df1d-428d-ab61-a1699d8bcbb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '664503abad3148b3acb9a51af502f8d2'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:45.162112 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5bbc4495-df1d-428d-ab61-a1699d8bcbb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '664503abad3148b3acb9a51af502f8d2'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:45.162830 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5bbc4495-df1d-428d-ab61-a1699d8bcbb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '664503abad3148b3acb9a51af502f8d2'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:45.176296 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1057/2103] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:18:45 2022] DELETE /identity/v3/users/664503abad3148b3acb9a51af502f8d2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:45.560061 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:45.562296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:45.612060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25c4f4-1717-423d-9f91-389e0cdfa7cf admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:45.691575 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd2de3-a90a-422e-b0fa-9ce5d4c7bb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:45.691575 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd2de3-a90a-422e-b0fa-9ce5d4c7bb01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:45.691575 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd2de3-a90a-422e-b0fa-9ce5d4c7bb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:45.691575 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd2de3-a90a-422e-b0fa-9ce5d4c7bb01 admin admin] PATH_INFO: `/v3/projects/bde03995f7744cd9b6493b5f4fbbe587` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:45.691896 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd2de3-a90a-422e-b0fa-9ce5d4c7bb01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bde03995f7744cd9b6493b5f4fbbe587)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:45.692820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd2de3-a90a-422e-b0fa-9ce5d4c7bb01 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:45.755349 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-fbdd2de3-a90a-422e-b0fa-9ce5d4c7bb01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bde03995f7744cd9b6493b5f4fbbe587', 'request_id': 'req-fbdd2de3-a90a-422e-b0fa-9ce5d4c7bb01'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:45.766914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-fbdd2de3-a90a-422e-b0fa-9ce5d4c7bb01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bde03995f7744cd9b6493b5f4fbbe587', 'request_id': 'req-fbdd2de3-a90a-422e-b0fa-9ce5d4c7bb01'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:45.775041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-fbdd2de3-a90a-422e-b0fa-9ce5d4c7bb01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bde03995f7744cd9b6493b5f4fbbe587', 'request_id': 'req-fbdd2de3-a90a-422e-b0fa-9ce5d4c7bb01'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:45.791410 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-fbdd2de3-a90a-422e-b0fa-9ce5d4c7bb01 admin admin] The token cache is being invalidate because project bde03995f7744cd9b6493b5f4fbbe587 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:45.791766 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-fbdd2de3-a90a-422e-b0fa-9ce5d4c7bb01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:45.792461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-fbdd2de3-a90a-422e-b0fa-9ce5d4c7bb01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:45.794107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1047/2104] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:45 2022] DELETE /identity/v3/projects/bde03995f7744cd9b6493b5f4fbbe587 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:45.798828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5bda1-0473-416f-8e23-78f1071c3da8 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:45.799112 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5bda1-0473-416f-8e23-78f1071c3da8 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:45.799388 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f5bda1-0473-416f-8e23-78f1071c3da8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:45.819771 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-68f5bda1-0473-416f-8e23-78f1071c3da8 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:45.821187 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-68f5bda1-0473-416f-8e23-78f1071c3da8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:45.839870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:45.896759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1058/2105] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:45.904220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533755d-6ce7-43e1-b862-f240d1d26e62 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:45.904524 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533755d-6ce7-43e1-b862-f240d1d26e62 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:45.904870 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7533755d-6ce7-43e1-b862-f240d1d26e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:45.928350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-7533755d-6ce7-43e1-b862-f240d1d26e62 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:45.929603 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7533755d-6ce7-43e1-b862-f240d1d26e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:45.939746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1048/2106] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:45.944140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5bda1-0473-416f-8e23-78f1071c3da8 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:45.955525 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f5bda1-0473-416f-8e23-78f1071c3da8 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:45.961382 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e72f78-9372-44d2-b02f-92e8333cf978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:45.962001 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e72f78-9372-44d2-b02f-92e8333cf978 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:45.962286 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e72f78-9372-44d2-b02f-92e8333cf978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:45.962557 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e72f78-9372-44d2-b02f-92e8333cf978 admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:45.963376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e72f78-9372-44d2-b02f-92e8333cf978 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:18:45.963543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e72f78-9372-44d2-b02f-92e8333cf978 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:45.965164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e72f78-9372-44d2-b02f-92e8333cf978 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:45.978265 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1059/2107] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 04 02:18:45.982549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533755d-6ce7-43e1-b862-f240d1d26e62 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:45.992039 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7533755d-6ce7-43e1-b862-f240d1d26e62 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:45.997800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42e6e2-da5e-484e-b27f-2fc17f6579bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:45.998520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42e6e2-da5e-484e-b27f-2fc17f6579bd admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:45.998895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42e6e2-da5e-484e-b27f-2fc17f6579bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:45.999368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e42e6e2-da5e-484e-b27f-2fc17f6579bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:46.000110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42e6e2-da5e-484e-b27f-2fc17f6579bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:46.002388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e42e6e2-da5e-484e-b27f-2fc17f6579bd admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:46.033148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1049/2108] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:18:45 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:46.038267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e72f78-9372-44d2-b02f-92e8333cf978 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:46.049097 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e72f78-9372-44d2-b02f-92e8333cf978 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:46.116999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593be75-585a-49b9-ad17-2995e2674b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:46.118022 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593be75-585a-49b9-ad17-2995e2674b4d admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:46.118302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593be75-585a-49b9-ad17-2995e2674b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:46.118574 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593be75-585a-49b9-ad17-2995e2674b4d admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:46.119262 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593be75-585a-49b9-ad17-2995e2674b4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:46.121988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593be75-585a-49b9-ad17-2995e2674b4d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:46.147940 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1060/2109] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:18:46 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:46.153064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42e6e2-da5e-484e-b27f-2fc17f6579bd admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:46.170436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e42e6e2-da5e-484e-b27f-2fc17f6579bd admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:46.175437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bad7d-0314-4f1c-a0fc-55fa565ef90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:46.176062 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bad7d-0314-4f1c-a0fc-55fa565ef90a admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:46.176395 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bad7d-0314-4f1c-a0fc-55fa565ef90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:46.176640 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bad7d-0314-4f1c-a0fc-55fa565ef90a admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:46.177349 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8bad7d-0314-4f1c-a0fc-55fa565ef90a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:46.178977 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8bad7d-0314-4f1c-a0fc-55fa565ef90a admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:46.190314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1050/2110] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:46.194205 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593be75-585a-49b9-ad17-2995e2674b4d admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:46.205108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593be75-585a-49b9-ad17-2995e2674b4d admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:46.210319 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fd3a59-5ff7-4beb-b130-7fcdc8ab8fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:46.210924 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd3a59-5ff7-4beb-b130-7fcdc8ab8fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:46.211916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd3a59-5ff7-4beb-b130-7fcdc8ab8fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:46.212193 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fd3a59-5ff7-4beb-b130-7fcdc8ab8fd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:46.212923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd3a59-5ff7-4beb-b130-7fcdc8ab8fd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:46.214530 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fd3a59-5ff7-4beb-b130-7fcdc8ab8fd0 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:46.225507 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1061/2111] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:46.230555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bad7d-0314-4f1c-a0fc-55fa565ef90a admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:46.240495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bad7d-0314-4f1c-a0fc-55fa565ef90a admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:46.245407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596ff40-c8fd-4f27-83fd-2949cd72aae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:46.246074 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596ff40-c8fd-4f27-83fd-2949cd72aae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:46.246355 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596ff40-c8fd-4f27-83fd-2949cd72aae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:46.246656 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596ff40-c8fd-4f27-83fd-2949cd72aae5 admin admin] PATH_INFO: `/v3/projects/e173117d16a24d84bbb30394b7f06503/users/7c64ce2110474affab6164e5fcbfc43f/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:46.261645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596ff40-c8fd-4f27-83fd-2949cd72aae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e173117d16a24d84bbb30394b7f06503, user_id=7c64ce2110474affab6164e5fcbfc43f, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:46.264446 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c596ff40-c8fd-4f27-83fd-2949cd72aae5 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:46.273396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:46.279633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1051/2112] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:46 2022] PUT /identity/v3/projects/e173117d16a24d84bbb30394b7f06503/users/7c64ce2110474affab6164e5fcbfc43f/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:46.290560 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca19ca-87f4-464f-b511-ca4a4dc4866f None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:46.290841 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca19ca-87f4-464f-b511-ca4a4dc4866f None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:46.291117 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca19ca-87f4-464f-b511-ca4a4dc4866f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:46.318612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-6bca19ca-87f4-464f-b511-ca4a4dc4866f None None] MFA Rules not processed for user `7c64ce2110474affab6164e5fcbfc43f`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:46.319820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-6bca19ca-87f4-464f-b511-ca4a4dc4866f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:46.389343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1062/2113] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:46.393938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596ff40-c8fd-4f27-83fd-2949cd72aae5 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:46.455758 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596ff40-c8fd-4f27-83fd-2949cd72aae5 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:46.461053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cbdc9-4792-448f-acf5-ca48e182a0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:46.462276 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cbdc9-4792-448f-acf5-ca48e182a0c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:46.462557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cbdc9-4792-448f-acf5-ca48e182a0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:46.462927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cbdc9-4792-448f-acf5-ca48e182a0c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:46.463676 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6cbdc9-4792-448f-acf5-ca48e182a0c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:46.465614 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6cbdc9-4792-448f-acf5-ca48e182a0c8 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:46.494623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1052/2114] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:18:46 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:46.495823 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca19ca-87f4-464f-b511-ca4a4dc4866f None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:46.506438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca19ca-87f4-464f-b511-ca4a4dc4866f None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:46.519436 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:46.568509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db502b-d725-493f-8aa7-8606b7fb6e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:46.569204 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db502b-d725-493f-8aa7-8606b7fb6e09 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:46.569512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db502b-d725-493f-8aa7-8606b7fb6e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:46.569777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db502b-d725-493f-8aa7-8606b7fb6e09 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:46.570451 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db502b-d725-493f-8aa7-8606b7fb6e09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:46.572933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db502b-d725-493f-8aa7-8606b7fb6e09 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:46.597094 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1063/2115] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:18:46 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:46.600993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cbdc9-4792-448f-acf5-ca48e182a0c8 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:46.611630 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cbdc9-4792-448f-acf5-ca48e182a0c8 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:46.617156 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536ea3c-bb3b-49ca-b80c-18430cf185e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:46.618621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536ea3c-bb3b-49ca-b80c-18430cf185e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:46.618621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536ea3c-bb3b-49ca-b80c-18430cf185e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:46.618892 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536ea3c-bb3b-49ca-b80c-18430cf185e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:46.619660 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536ea3c-bb3b-49ca-b80c-18430cf185e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:46.621332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536ea3c-bb3b-49ca-b80c-18430cf185e2 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:46.637531 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1053/2116] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:46.639149 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db502b-d725-493f-8aa7-8606b7fb6e09 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:46.672912 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db502b-d725-493f-8aa7-8606b7fb6e09 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:46.681558 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c257c1c-973f-46a1-89e2-bbabf7928580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:46.683112 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c257c1c-973f-46a1-89e2-bbabf7928580 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:46.683500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c257c1c-973f-46a1-89e2-bbabf7928580 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:46.683875 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c257c1c-973f-46a1-89e2-bbabf7928580 admin admin] PATH_INFO: `/v3/projects/12b286c70c914b1a82c2ba1f0fbb3a03/users/03962ea9ecaf4855996879a4e1323050/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:46.721216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c257c1c-973f-46a1-89e2-bbabf7928580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b286c70c914b1a82c2ba1f0fbb3a03, user_id=03962ea9ecaf4855996879a4e1323050, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:46.727808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c257c1c-973f-46a1-89e2-bbabf7928580 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:46.758141 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1064/2117] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:46 2022] PUT /identity/v3/projects/12b286c70c914b1a82c2ba1f0fbb3a03/users/03962ea9ecaf4855996879a4e1323050/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:46.762443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536ea3c-bb3b-49ca-b80c-18430cf185e2 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:46.774380 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536ea3c-bb3b-49ca-b80c-18430cf185e2 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:46.860872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c257c1c-973f-46a1-89e2-bbabf7928580 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:46.863563 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-6c257c1c-973f-46a1-89e2-bbabf7928580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:46.870439 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c58916-eae1-4924-bb60-4682e776cc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:46.871381 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c58916-eae1-4924-bb60-4682e776cc58 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:46.871810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c58916-eae1-4924-bb60-4682e776cc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:46.872227 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c58916-eae1-4924-bb60-4682e776cc58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:46.873126 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c58916-eae1-4924-bb60-4682e776cc58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:46.874926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c58916-eae1-4924-bb60-4682e776cc58 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:46.888416 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1054/2118] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:46.892102 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c58916-eae1-4924-bb60-4682e776cc58 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:46.904159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c58916-eae1-4924-bb60-4682e776cc58 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:46.909651 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49abb7c5-b9d5-424e-8a45-f12ba0f955d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:46.910316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abb7c5-b9d5-424e-8a45-f12ba0f955d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:46.910743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abb7c5-b9d5-424e-8a45-f12ba0f955d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:46.911001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49abb7c5-b9d5-424e-8a45-f12ba0f955d5 admin admin] PATH_INFO: `/v3/projects/12b286c70c914b1a82c2ba1f0fbb3a03/users/03962ea9ecaf4855996879a4e1323050/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:46.913255 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49abb7c5-b9d5-424e-8a45-f12ba0f955d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b286c70c914b1a82c2ba1f0fbb3a03, user_id=03962ea9ecaf4855996879a4e1323050, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:46.916798 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49abb7c5-b9d5-424e-8a45-f12ba0f955d5 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:46.928938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1055/2119] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:46 2022] PUT /identity/v3/projects/12b286c70c914b1a82c2ba1f0fbb3a03/users/03962ea9ecaf4855996879a4e1323050/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:18:46.940957 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260d442-1324-4789-9cae-708ca77e8104 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:46.941295 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260d442-1324-4789-9cae-708ca77e8104 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:46.941576 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260d442-1324-4789-9cae-708ca77e8104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:46.947370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c257c1c-973f-46a1-89e2-bbabf7928580 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:46.974176 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-f260d442-1324-4789-9cae-708ca77e8104 None None] MFA Rules not processed for user `03962ea9ecaf4855996879a4e1323050`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:46.975463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-f260d442-1324-4789-9cae-708ca77e8104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:47.025552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47334696-d86b-4a48-8e43-2a481e75dd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:47.026580 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47334696-d86b-4a48-8e43-2a481e75dd1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:47.027049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47334696-d86b-4a48-8e43-2a481e75dd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:47.027474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47334696-d86b-4a48-8e43-2a481e75dd1b admin admin] PATH_INFO: `/v3/users/cca6c55bfff843ab8c4c3d007e3a2ffc` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:47.028981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47334696-d86b-4a48-8e43-2a481e75dd1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cca6c55bfff843ab8c4c3d007e3a2ffc)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:47.031415 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47334696-d86b-4a48-8e43-2a481e75dd1b admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:47.032291 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:47.077871 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1056/2120] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:18:47.079237 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:47.089445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260d442-1324-4789-9cae-708ca77e8104 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:47.091024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-f260d442-1324-4789-9cae-708ca77e8104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:47.101047 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-47334696-d86b-4a48-8e43-2a481e75dd1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cca6c55bfff843ab8c4c3d007e3a2ffc'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:47.110248 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-47334696-d86b-4a48-8e43-2a481e75dd1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cca6c55bfff843ab8c4c3d007e3a2ffc'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:47.111128 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-47334696-d86b-4a48-8e43-2a481e75dd1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cca6c55bfff843ab8c4c3d007e3a2ffc'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:47.127779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-47334696-d86b-4a48-8e43-2a481e75dd1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cca6c55bfff843ab8c4c3d007e3a2ffc'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:47.129067 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-47334696-d86b-4a48-8e43-2a481e75dd1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cca6c55bfff843ab8c4c3d007e3a2ffc'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:47.153843 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1065/2121] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:18:46 2022] DELETE /identity/v3/users/cca6c55bfff843ab8c4c3d007e3a2ffc => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:18:47.155633 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13498e-9771-476c-8b8e-3f9229e58428 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:47.155893 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13498e-9771-476c-8b8e-3f9229e58428 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:47.156176 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df13498e-9771-476c-8b8e-3f9229e58428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:47.162166 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260d442-1324-4789-9cae-708ca77e8104 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:47.184904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-df13498e-9771-476c-8b8e-3f9229e58428 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:47.186512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-df13498e-9771-476c-8b8e-3f9229e58428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:47.236407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4dfc79-e43e-49f9-9f59-3cde05db0601 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:47.237155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dfc79-e43e-49f9-9f59-3cde05db0601 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:47.237747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dfc79-e43e-49f9-9f59-3cde05db0601 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:47.238092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4dfc79-e43e-49f9-9f59-3cde05db0601 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:47.241740 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-5a4dfc79-e43e-49f9-9f59-3cde05db0601 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:47.267362 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1066/2122] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:18:47.278595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13498e-9771-476c-8b8e-3f9229e58428 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:47.280534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-df13498e-9771-476c-8b8e-3f9229e58428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:47.365680 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4dfc79-e43e-49f9-9f59-3cde05db0601 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:47.367442 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df13498e-9771-476c-8b8e-3f9229e58428 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:47.371339 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4dfc79-e43e-49f9-9f59-3cde05db0601 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:47.372601 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b535381-464a-4eeb-8587-b6b4ad3716ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:47.375824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b535381-464a-4eeb-8587-b6b4ad3716ba service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:47.376193 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b535381-464a-4eeb-8587-b6b4ad3716ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:47.376601 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b535381-464a-4eeb-8587-b6b4ad3716ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:47.398488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1057/2123] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:18:47.399063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b535381-464a-4eeb-8587-b6b4ad3716ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:47.400975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b535381-464a-4eeb-8587-b6b4ad3716ba service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:47.432833 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1067/2124] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:47.513679 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4dfc79-e43e-49f9-9f59-3cde05db0601 service cinder] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:47.515211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-5a4dfc79-e43e-49f9-9f59-3cde05db0601 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:47.527596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4dfc79-e43e-49f9-9f59-3cde05db0601 service cinder] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:47.531456 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71643706-21cd-4651-a858-313c3c50503e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:47.532075 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71643706-21cd-4651-a858-313c3c50503e service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:47.532361 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71643706-21cd-4651-a858-313c3c50503e service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:47.532655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71643706-21cd-4651-a858-313c3c50503e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:47.534447 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-71643706-21cd-4651-a858-313c3c50503e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:47.594177 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:47.611764 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71643706-21cd-4651-a858-313c3c50503e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:47.613512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71643706-21cd-4651-a858-313c3c50503e service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:47.632843 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1058/2125] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:47.954009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b535381-464a-4eeb-8587-b6b4ad3716ba service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:47.955465 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8b535381-464a-4eeb-8587-b6b4ad3716ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:47.969519 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b535381-464a-4eeb-8587-b6b4ad3716ba service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:47.983130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a337090-c3fb-4e11-a466-23be7193fb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:47.983931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a337090-c3fb-4e11-a466-23be7193fb12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:47.984214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a337090-c3fb-4e11-a466-23be7193fb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:47.984545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a337090-c3fb-4e11-a466-23be7193fb12 admin admin] PATH_INFO: `/v3/projects/48536d0e5f39411792d524f349eda2b8` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:47.986254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a337090-c3fb-4e11-a466-23be7193fb12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48536d0e5f39411792d524f349eda2b8)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:47.990391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a337090-c3fb-4e11-a466-23be7193fb12 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:48.047648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:48.049205 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1a337090-c3fb-4e11-a466-23be7193fb12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48536d0e5f39411792d524f349eda2b8', 'request_id': 'req-1a337090-c3fb-4e11-a466-23be7193fb12'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.057932 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1a337090-c3fb-4e11-a466-23be7193fb12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48536d0e5f39411792d524f349eda2b8', 'request_id': 'req-1a337090-c3fb-4e11-a466-23be7193fb12'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.064929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1a337090-c3fb-4e11-a466-23be7193fb12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48536d0e5f39411792d524f349eda2b8', 'request_id': 'req-1a337090-c3fb-4e11-a466-23be7193fb12'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.077992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1a337090-c3fb-4e11-a466-23be7193fb12 admin admin] The token cache is being invalidate because project 48536d0e5f39411792d524f349eda2b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:48.078288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1a337090-c3fb-4e11-a466-23be7193fb12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.078989 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1a337090-c3fb-4e11-a466-23be7193fb12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.080655 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1068/2126] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:47 2022] DELETE /identity/v3/projects/48536d0e5f39411792d524f349eda2b8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:48.085020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71643706-21cd-4651-a858-313c3c50503e service cinder] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:48.095594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71643706-21cd-4651-a858-313c3c50503e service cinder] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:48.170247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af469d8-2a17-46e0-841e-88002158378d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:48.170954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af469d8-2a17-46e0-841e-88002158378d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:48.171236 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af469d8-2a17-46e0-841e-88002158378d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:48.171509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af469d8-2a17-46e0-841e-88002158378d admin admin] PATH_INFO: `/v3/users/0b3f78e7c03a44e8967bb6702a9e2e2a` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:48.172729 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af469d8-2a17-46e0-841e-88002158378d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b3f78e7c03a44e8967bb6702a9e2e2a)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:48.174693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af469d8-2a17-46e0-841e-88002158378d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:48.227449 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-6af469d8-2a17-46e0-841e-88002158378d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b3f78e7c03a44e8967bb6702a9e2e2a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.228288 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-6af469d8-2a17-46e0-841e-88002158378d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b3f78e7c03a44e8967bb6702a9e2e2a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.229660 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-6af469d8-2a17-46e0-841e-88002158378d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b3f78e7c03a44e8967bb6702a9e2e2a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.249982 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-6af469d8-2a17-46e0-841e-88002158378d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b3f78e7c03a44e8967bb6702a9e2e2a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.278045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-6af469d8-2a17-46e0-841e-88002158378d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b3f78e7c03a44e8967bb6702a9e2e2a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.298559 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1059/2127] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:18:48 2022] DELETE /identity/v3/users/0b3f78e7c03a44e8967bb6702a9e2e2a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:18:48.461409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a337090-c3fb-4e11-a466-23be7193fb12 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:48.462874 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1a337090-c3fb-4e11-a466-23be7193fb12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:48.479075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a337090-c3fb-4e11-a466-23be7193fb12 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:48.554573 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d943c2-0fc6-4ad5-8c49-9128c0d84af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:48.555922 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d943c2-0fc6-4ad5-8c49-9128c0d84af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:48.556208 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d943c2-0fc6-4ad5-8c49-9128c0d84af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:48.557013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d943c2-0fc6-4ad5-8c49-9128c0d84af2 admin admin] PATH_INFO: `/v3/users/d4a9fa95ae1245a3b705d3d181146451` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:48.558281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d943c2-0fc6-4ad5-8c49-9128c0d84af2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4a9fa95ae1245a3b705d3d181146451)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:48.560241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d943c2-0fc6-4ad5-8c49-9128c0d84af2 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:48.598269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af469d8-2a17-46e0-841e-88002158378d admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:48.601522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-6af469d8-2a17-46e0-841e-88002158378d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:48.618762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af469d8-2a17-46e0-841e-88002158378d admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:48.627125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:48.628526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b5d943c2-0fc6-4ad5-8c49-9128c0d84af2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4a9fa95ae1245a3b705d3d181146451'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.629623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:48.630012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:48.631077 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1 admin admin] PATH_INFO: `/v3/projects/11bd4bf74ace4feeb93ff91cfc90a496` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:48.633407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11bd4bf74ace4feeb93ff91cfc90a496)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:48.638965 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:48.640777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b5d943c2-0fc6-4ad5-8c49-9128c0d84af2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4a9fa95ae1245a3b705d3d181146451'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.642383 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b5d943c2-0fc6-4ad5-8c49-9128c0d84af2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4a9fa95ae1245a3b705d3d181146451'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.663989 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b5d943c2-0fc6-4ad5-8c49-9128c0d84af2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4a9fa95ae1245a3b705d3d181146451'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.665289 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b5d943c2-0fc6-4ad5-8c49-9128c0d84af2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4a9fa95ae1245a3b705d3d181146451'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.681968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1069/2128] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:18:48 2022] DELETE /identity/v3/users/d4a9fa95ae1245a3b705d3d181146451 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:48.686231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808fb8c-e8ed-4a68-9a9b-97071bf9d84a None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:48.686544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808fb8c-e8ed-4a68-9a9b-97071bf9d84a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:48.686804 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4808fb8c-e8ed-4a68-9a9b-97071bf9d84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:48.714140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-4808fb8c-e8ed-4a68-9a9b-97071bf9d84a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:48.715502 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4808fb8c-e8ed-4a68-9a9b-97071bf9d84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:48.737625 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11bd4bf74ace4feeb93ff91cfc90a496', 'request_id': 'req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.745874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11bd4bf74ace4feeb93ff91cfc90a496', 'request_id': 'req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.751461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:48.754290 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11bd4bf74ace4feeb93ff91cfc90a496', 'request_id': 'req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.770333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1 admin admin] The token cache is being invalidate because project 11bd4bf74ace4feeb93ff91cfc90a496 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:48.770583 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.771229 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.771367 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:48.772902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1060/2129] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:48 2022] DELETE /identity/v3/projects/11bd4bf74ace4feeb93ff91cfc90a496 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:48.777548 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:48.778922 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:48.788844 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1070/2130] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:48.800294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808fb8c-e8ed-4a68-9a9b-97071bf9d84a None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:48.801845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4808fb8c-e8ed-4a68-9a9b-97071bf9d84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:48.848009 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4b5ead-ccfb-4939-af2d-3fc7dc297cc1 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:48.853885 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af387ce1-4625-46b1-a8e1-346e98f4066b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:48.854607 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af387ce1-4625-46b1-a8e1-346e98f4066b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:48.854808 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af387ce1-4625-46b1-a8e1-346e98f4066b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:48.855078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af387ce1-4625-46b1-a8e1-346e98f4066b admin admin] PATH_INFO: `/v3/users/ab21f5ab4d154eb2881e920660c7e588` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:48.856648 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af387ce1-4625-46b1-a8e1-346e98f4066b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab21f5ab4d154eb2881e920660c7e588)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:48.858230 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af387ce1-4625-46b1-a8e1-346e98f4066b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:48.881862 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4808fb8c-e8ed-4a68-9a9b-97071bf9d84a None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:48.916564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-af387ce1-4625-46b1-a8e1-346e98f4066b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab21f5ab4d154eb2881e920660c7e588'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.920079 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-af387ce1-4625-46b1-a8e1-346e98f4066b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab21f5ab4d154eb2881e920660c7e588'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.920829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-af387ce1-4625-46b1-a8e1-346e98f4066b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab21f5ab4d154eb2881e920660c7e588'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.942569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-af387ce1-4625-46b1-a8e1-346e98f4066b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab21f5ab4d154eb2881e920660c7e588'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.952312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2989aeb-40fe-4710-953a-e92d16a77c1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:48.952975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2989aeb-40fe-4710-953a-e92d16a77c1b service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:48.953258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2989aeb-40fe-4710-953a-e92d16a77c1b service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:48.953537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2989aeb-40fe-4710-953a-e92d16a77c1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:48.953741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-af387ce1-4625-46b1-a8e1-346e98f4066b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab21f5ab4d154eb2881e920660c7e588'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:48.955483 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f2989aeb-40fe-4710-953a-e92d16a77c1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:48.970217 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2989aeb-40fe-4710-953a-e92d16a77c1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:48.971728 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2989aeb-40fe-4710-953a-e92d16a77c1b service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:48.972909 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1061/2131] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:18:48 2022] DELETE /identity/v3/users/ab21f5ab4d154eb2881e920660c7e588 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:18:48.984597 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1071/2132] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:18:49.288529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af387ce1-4625-46b1-a8e1-346e98f4066b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:49.290623 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-af387ce1-4625-46b1-a8e1-346e98f4066b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:49.430541 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af387ce1-4625-46b1-a8e1-346e98f4066b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:49.442787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753ec9e-74d1-4526-8836-b62ad090d36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:49.443876 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753ec9e-74d1-4526-8836-b62ad090d36d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:49.444289 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753ec9e-74d1-4526-8836-b62ad090d36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:49.444906 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753ec9e-74d1-4526-8836-b62ad090d36d admin admin] PATH_INFO: `/v3/projects/254dd7e132eb4feb80dc592337000fe8` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:49.447626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753ec9e-74d1-4526-8836-b62ad090d36d admin admin] RBAC: Authorizing `identity:delete_project(project_id=254dd7e132eb4feb80dc592337000fe8)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:49.451151 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753ec9e-74d1-4526-8836-b62ad090d36d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:49.464266 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:49.517539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d753ec9e-74d1-4526-8836-b62ad090d36d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '254dd7e132eb4feb80dc592337000fe8', 'request_id': 'req-d753ec9e-74d1-4526-8836-b62ad090d36d'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:49.532573 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d753ec9e-74d1-4526-8836-b62ad090d36d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '254dd7e132eb4feb80dc592337000fe8', 'request_id': 'req-d753ec9e-74d1-4526-8836-b62ad090d36d'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:49.539912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d753ec9e-74d1-4526-8836-b62ad090d36d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '254dd7e132eb4feb80dc592337000fe8', 'request_id': 'req-d753ec9e-74d1-4526-8836-b62ad090d36d'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:49.553319 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d753ec9e-74d1-4526-8836-b62ad090d36d admin admin] The token cache is being invalidate because project 254dd7e132eb4feb80dc592337000fe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:49.553743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d753ec9e-74d1-4526-8836-b62ad090d36d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:49.554800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d753ec9e-74d1-4526-8836-b62ad090d36d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:49.559010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1062/2133] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:49 2022] DELETE /identity/v3/projects/254dd7e132eb4feb80dc592337000fe8 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:49.617157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2989aeb-40fe-4710-953a-e92d16a77c1b service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:49.622219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f2989aeb-40fe-4710-953a-e92d16a77c1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:49.651499 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2989aeb-40fe-4710-953a-e92d16a77c1b service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:49.731215 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44304dc4-a912-4a35-9d9d-5f4b4e774980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:49.731952 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44304dc4-a912-4a35-9d9d-5f4b4e774980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:49.732273 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44304dc4-a912-4a35-9d9d-5f4b4e774980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:49.732563 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44304dc4-a912-4a35-9d9d-5f4b4e774980 admin admin] PATH_INFO: `/v3/projects/b5150eed305142bc9cf3a5dfb48d2583` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:49.734106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44304dc4-a912-4a35-9d9d-5f4b4e774980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5150eed305142bc9cf3a5dfb48d2583)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:49.736117 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44304dc4-a912-4a35-9d9d-5f4b4e774980 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:49.794032 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-44304dc4-a912-4a35-9d9d-5f4b4e774980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5150eed305142bc9cf3a5dfb48d2583', 'request_id': 'req-44304dc4-a912-4a35-9d9d-5f4b4e774980'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:49.802547 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-44304dc4-a912-4a35-9d9d-5f4b4e774980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5150eed305142bc9cf3a5dfb48d2583', 'request_id': 'req-44304dc4-a912-4a35-9d9d-5f4b4e774980'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:49.804006 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:49.811450 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-44304dc4-a912-4a35-9d9d-5f4b4e774980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5150eed305142bc9cf3a5dfb48d2583', 'request_id': 'req-44304dc4-a912-4a35-9d9d-5f4b4e774980'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:49.825766 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-44304dc4-a912-4a35-9d9d-5f4b4e774980 admin admin] The token cache is being invalidate because project b5150eed305142bc9cf3a5dfb48d2583 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:49.826065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-44304dc4-a912-4a35-9d9d-5f4b4e774980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:49.826783 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-44304dc4-a912-4a35-9d9d-5f4b4e774980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:49.828322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1072/2134] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:49 2022] DELETE /identity/v3/projects/b5150eed305142bc9cf3a5dfb48d2583 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:57.113001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753ec9e-74d1-4526-8836-b62ad090d36d admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:57.114447 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d753ec9e-74d1-4526-8836-b62ad090d36d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:57.127920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753ec9e-74d1-4526-8836-b62ad090d36d admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:57.185003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b981df2-d74c-42fd-a534-54ee938a9a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:57.185546 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b981df2-d74c-42fd-a534-54ee938a9a9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:57.185819 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b981df2-d74c-42fd-a534-54ee938a9a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:57.186088 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b981df2-d74c-42fd-a534-54ee938a9a9c admin admin] PATH_INFO: `/v3/users/7c64ce2110474affab6164e5fcbfc43f` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:57.187228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b981df2-d74c-42fd-a534-54ee938a9a9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c64ce2110474affab6164e5fcbfc43f)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:57.189292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b981df2-d74c-42fd-a534-54ee938a9a9c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:57.234891 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b981df2-d74c-42fd-a534-54ee938a9a9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c64ce2110474affab6164e5fcbfc43f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:57.235531 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b981df2-d74c-42fd-a534-54ee938a9a9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c64ce2110474affab6164e5fcbfc43f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:57.236230 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b981df2-d74c-42fd-a534-54ee938a9a9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c64ce2110474affab6164e5fcbfc43f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:57.248428 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b981df2-d74c-42fd-a534-54ee938a9a9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c64ce2110474affab6164e5fcbfc43f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:57.255205 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b981df2-d74c-42fd-a534-54ee938a9a9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c64ce2110474affab6164e5fcbfc43f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:57.268417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1063/2135] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:18:57 2022] DELETE /identity/v3/users/7c64ce2110474affab6164e5fcbfc43f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:57.273040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806c7a4-9be8-4e92-bf09-942b8bcebba9 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:57.273514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806c7a4-9be8-4e92-bf09-942b8bcebba9 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:57.273607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806c7a4-9be8-4e92-bf09-942b8bcebba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:57.293214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c806c7a4-9be8-4e92-bf09-942b8bcebba9 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:57.294398 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c806c7a4-9be8-4e92-bf09-942b8bcebba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:57.354675 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1073/2136] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:57.364590 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b981df2-d74c-42fd-a534-54ee938a9a9c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:57.366024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-8b981df2-d74c-42fd-a534-54ee938a9a9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:57.386667 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:57.427147 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b981df2-d74c-42fd-a534-54ee938a9a9c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:57.430750 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ded38-0a5d-47ab-97a9-9b46a3c23626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:57.431406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ded38-0a5d-47ab-97a9-9b46a3c23626 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:57.431681 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ded38-0a5d-47ab-97a9-9b46a3c23626 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:57.431946 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506ded38-0a5d-47ab-97a9-9b46a3c23626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:57.440901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ded38-0a5d-47ab-97a9-9b46a3c23626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:57.442238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506ded38-0a5d-47ab-97a9-9b46a3c23626 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:57.451600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1064/2137] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:57.721150 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806c7a4-9be8-4e92-bf09-942b8bcebba9 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:57.722595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c806c7a4-9be8-4e92-bf09-942b8bcebba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:57.738889 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806c7a4-9be8-4e92-bf09-942b8bcebba9 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:57.745695 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb87d85-18af-4603-b5cc-e247e480dc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:57.746419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb87d85-18af-4603-b5cc-e247e480dc4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:57.746619 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb87d85-18af-4603-b5cc-e247e480dc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:57.747205 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb87d85-18af-4603-b5cc-e247e480dc4a admin admin] PATH_INFO: `/v3/projects/e173117d16a24d84bbb30394b7f06503` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:57.748678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb87d85-18af-4603-b5cc-e247e480dc4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e173117d16a24d84bbb30394b7f06503)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:57.750566 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb87d85-18af-4603-b5cc-e247e480dc4a admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:57.797427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fdb87d85-18af-4603-b5cc-e247e480dc4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e173117d16a24d84bbb30394b7f06503', 'request_id': 'req-fdb87d85-18af-4603-b5cc-e247e480dc4a'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:57.804582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fdb87d85-18af-4603-b5cc-e247e480dc4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e173117d16a24d84bbb30394b7f06503', 'request_id': 'req-fdb87d85-18af-4603-b5cc-e247e480dc4a'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:57.811081 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fdb87d85-18af-4603-b5cc-e247e480dc4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e173117d16a24d84bbb30394b7f06503', 'request_id': 'req-fdb87d85-18af-4603-b5cc-e247e480dc4a'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:57.821878 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fdb87d85-18af-4603-b5cc-e247e480dc4a admin admin] The token cache is being invalidate because project e173117d16a24d84bbb30394b7f06503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:57.822164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fdb87d85-18af-4603-b5cc-e247e480dc4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:57.822820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fdb87d85-18af-4603-b5cc-e247e480dc4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:57.824313 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1074/2138] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:57 2022] DELETE /identity/v3/projects/e173117d16a24d84bbb30394b7f06503 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:18:57.828291 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ded38-0a5d-47ab-97a9-9b46a3c23626 service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:57.838681 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506ded38-0a5d-47ab-97a9-9b46a3c23626 service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:57.907975 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe24e6-59c9-4d86-8aab-e45280b18c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:57.908737 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe24e6-59c9-4d86-8aab-e45280b18c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:57.909017 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe24e6-59c9-4d86-8aab-e45280b18c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:57.909289 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe24e6-59c9-4d86-8aab-e45280b18c54 admin admin] PATH_INFO: `/v3/users/03962ea9ecaf4855996879a4e1323050` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:57.910936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe24e6-59c9-4d86-8aab-e45280b18c54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03962ea9ecaf4855996879a4e1323050)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:57.913168 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe24e6-59c9-4d86-8aab-e45280b18c54 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:57.963448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e9fe24e6-59c9-4d86-8aab-e45280b18c54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03962ea9ecaf4855996879a4e1323050'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:57.964301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e9fe24e6-59c9-4d86-8aab-e45280b18c54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03962ea9ecaf4855996879a4e1323050'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:57.964862 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e9fe24e6-59c9-4d86-8aab-e45280b18c54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03962ea9ecaf4855996879a4e1323050'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:57.991673 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e9fe24e6-59c9-4d86-8aab-e45280b18c54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03962ea9ecaf4855996879a4e1323050'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:58.000464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e9fe24e6-59c9-4d86-8aab-e45280b18c54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03962ea9ecaf4855996879a4e1323050'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:58.017596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1065/2139] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:18:57 2022] DELETE /identity/v3/users/03962ea9ecaf4855996879a4e1323050 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:58.324427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb87d85-18af-4603-b5cc-e247e480dc4a admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:58.325919 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-fdb87d85-18af-4603-b5cc-e247e480dc4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:58.340917 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb87d85-18af-4603-b5cc-e247e480dc4a admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:58.405240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c7d0c-32e4-4ca7-a989-24b071416d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:58.405882 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c7d0c-32e4-4ca7-a989-24b071416d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:58.406165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c7d0c-32e4-4ca7-a989-24b071416d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:58.406545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c7d0c-32e4-4ca7-a989-24b071416d76 admin admin] PATH_INFO: `/v3/projects/12b286c70c914b1a82c2ba1f0fbb3a03` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:58.407614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c7d0c-32e4-4ca7-a989-24b071416d76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b286c70c914b1a82c2ba1f0fbb3a03)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:58.409550 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c7d0c-32e4-4ca7-a989-24b071416d76 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:58.420503 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:58.457926 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-616c7d0c-32e4-4ca7-a989-24b071416d76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b286c70c914b1a82c2ba1f0fbb3a03', 'request_id': 'req-616c7d0c-32e4-4ca7-a989-24b071416d76'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:58.465043 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-616c7d0c-32e4-4ca7-a989-24b071416d76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b286c70c914b1a82c2ba1f0fbb3a03', 'request_id': 'req-616c7d0c-32e4-4ca7-a989-24b071416d76'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:58.470764 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-616c7d0c-32e4-4ca7-a989-24b071416d76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b286c70c914b1a82c2ba1f0fbb3a03', 'request_id': 'req-616c7d0c-32e4-4ca7-a989-24b071416d76'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:58.482454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-616c7d0c-32e4-4ca7-a989-24b071416d76 admin admin] The token cache is being invalidate because project 12b286c70c914b1a82c2ba1f0fbb3a03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:18:58.482678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-616c7d0c-32e4-4ca7-a989-24b071416d76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:58.483315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-616c7d0c-32e4-4ca7-a989-24b071416d76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:18:58.484849 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1075/2140] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:18:58 2022] DELETE /identity/v3/projects/12b286c70c914b1a82c2ba1f0fbb3a03 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:58.489811 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387fda8-9ce9-4d50-bfde-6c97ec992bbe None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:58.490301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387fda8-9ce9-4d50-bfde-6c97ec992bbe None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:58.490397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d387fda8-9ce9-4d50-bfde-6c97ec992bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:58.510501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-d387fda8-9ce9-4d50-bfde-6c97ec992bbe None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:58.511700 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d387fda8-9ce9-4d50-bfde-6c97ec992bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:58.548747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:58.578300 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1066/2141] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:58.586253 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161e0f4-b0b0-4c0b-a59e-d74339afe9b0 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:58.586543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161e0f4-b0b0-4c0b-a59e-d74339afe9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:58.586822 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d161e0f4-b0b0-4c0b-a59e-d74339afe9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:58.606868 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-d161e0f4-b0b0-4c0b-a59e-d74339afe9b0 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:58.608095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d161e0f4-b0b0-4c0b-a59e-d74339afe9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:58.616121 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1076/2142] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:58.621009 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387fda8-9ce9-4d50-bfde-6c97ec992bbe None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:58.630768 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d387fda8-9ce9-4d50-bfde-6c97ec992bbe None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:58.636672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6002d3d-fe31-449b-8562-57c5db804144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:58.637312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6002d3d-fe31-449b-8562-57c5db804144 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:58.637593 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6002d3d-fe31-449b-8562-57c5db804144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:58.637867 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6002d3d-fe31-449b-8562-57c5db804144 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:58.638616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6002d3d-fe31-449b-8562-57c5db804144 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:18:58.638921 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6002d3d-fe31-449b-8562-57c5db804144 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:58.641970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6002d3d-fe31-449b-8562-57c5db804144 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:58.656343 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1067/2143] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:18:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:18:58.661252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161e0f4-b0b0-4c0b-a59e-d74339afe9b0 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:58.671891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d161e0f4-b0b0-4c0b-a59e-d74339afe9b0 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:58.676506 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab56335-c9a6-4867-931b-88e528270f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:58.677326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab56335-c9a6-4867-931b-88e528270f00 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:58.677635 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab56335-c9a6-4867-931b-88e528270f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:58.677928 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab56335-c9a6-4867-931b-88e528270f00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:58.678589 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab56335-c9a6-4867-931b-88e528270f00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:58.680477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab56335-c9a6-4867-931b-88e528270f00 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:58.709466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1077/2144] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:18:58 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:58.714100 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6002d3d-fe31-449b-8562-57c5db804144 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:58.724382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6002d3d-fe31-449b-8562-57c5db804144 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:58.785465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf16546-08bd-4934-ae2e-7b4284533f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:58.785998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf16546-08bd-4934-ae2e-7b4284533f91 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:58.786306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf16546-08bd-4934-ae2e-7b4284533f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:58.786583 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf16546-08bd-4934-ae2e-7b4284533f91 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:58.787241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf16546-08bd-4934-ae2e-7b4284533f91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:58.789204 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf16546-08bd-4934-ae2e-7b4284533f91 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:58.810393 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1068/2145] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:18:58 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:58.814518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab56335-c9a6-4867-931b-88e528270f00 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:58.824853 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab56335-c9a6-4867-931b-88e528270f00 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:58.829467 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62057be2-1226-439c-973c-bbe77158a8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:58.830074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62057be2-1226-439c-973c-bbe77158a8fb admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:58.830356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62057be2-1226-439c-973c-bbe77158a8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:58.830631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62057be2-1226-439c-973c-bbe77158a8fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:58.831324 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62057be2-1226-439c-973c-bbe77158a8fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:58.832946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62057be2-1226-439c-973c-bbe77158a8fb admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:58.844258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1078/2146] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:58.848183 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf16546-08bd-4934-ae2e-7b4284533f91 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:58.858707 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf16546-08bd-4934-ae2e-7b4284533f91 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:58.863373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a032ae-27ae-4d4a-a83b-3f045f37e069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:58.863975 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a032ae-27ae-4d4a-a83b-3f045f37e069 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:58.864255 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a032ae-27ae-4d4a-a83b-3f045f37e069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:58.864528 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a032ae-27ae-4d4a-a83b-3f045f37e069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:58.865275 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a032ae-27ae-4d4a-a83b-3f045f37e069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:58.866859 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a032ae-27ae-4d4a-a83b-3f045f37e069 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:58.877403 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1069/2147] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:58.881438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62057be2-1226-439c-973c-bbe77158a8fb admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:58.891408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62057be2-1226-439c-973c-bbe77158a8fb admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:58.896073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df9fce-974a-4749-8790-0085a2abe59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:58.896759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df9fce-974a-4749-8790-0085a2abe59d admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:58.897043 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df9fce-974a-4749-8790-0085a2abe59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:58.897320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df9fce-974a-4749-8790-0085a2abe59d admin admin] PATH_INFO: `/v3/projects/bc0f3e996b574189a7a4359d1aa64c89/users/0818797b17e1444ea729d868f00ff1f4/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:58.912343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df9fce-974a-4749-8790-0085a2abe59d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc0f3e996b574189a7a4359d1aa64c89, user_id=0818797b17e1444ea729d868f00ff1f4, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:58.915050 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df9fce-974a-4749-8790-0085a2abe59d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:58.929140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1079/2148] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:58 2022] PUT /identity/v3/projects/bc0f3e996b574189a7a4359d1aa64c89/users/0818797b17e1444ea729d868f00ff1f4/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:18:58.940320 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50212a5-1286-4241-8aee-1791de164354 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:58.940610 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50212a5-1286-4241-8aee-1791de164354 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:58.940911 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50212a5-1286-4241-8aee-1791de164354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:58.969072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-b50212a5-1286-4241-8aee-1791de164354 None None] MFA Rules not processed for user `0818797b17e1444ea729d868f00ff1f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:58.970286 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-b50212a5-1286-4241-8aee-1791de164354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:59.034074 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1070/2149] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:59.039011 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df9fce-974a-4749-8790-0085a2abe59d admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:59.101770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df9fce-974a-4749-8790-0085a2abe59d admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:59.106992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce690687-64b7-4c25-9a75-716b3ee6f55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:59.107593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce690687-64b7-4c25-9a75-716b3ee6f55b admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:59.107878 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce690687-64b7-4c25-9a75-716b3ee6f55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:59.108155 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce690687-64b7-4c25-9a75-716b3ee6f55b admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:59.108839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce690687-64b7-4c25-9a75-716b3ee6f55b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:59.110726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce690687-64b7-4c25-9a75-716b3ee6f55b admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:59.137315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1080/2150] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:18:59 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:59.141488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50212a5-1286-4241-8aee-1791de164354 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:59.152971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50212a5-1286-4241-8aee-1791de164354 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:59.191734 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:59.220069 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3ed0a-b9c1-4017-adda-ec630c519895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:59.220706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3ed0a-b9c1-4017-adda-ec630c519895 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:59.221860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3ed0a-b9c1-4017-adda-ec630c519895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:59.222110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b3ed0a-b9c1-4017-adda-ec630c519895 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:59.222799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3ed0a-b9c1-4017-adda-ec630c519895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:59.224756 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b3ed0a-b9c1-4017-adda-ec630c519895 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:59.251368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1071/2151] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:18:59 2022] POST /identity/v3/users => generated 291 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:18:59.252529 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce690687-64b7-4c25-9a75-716b3ee6f55b admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:59.263378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce690687-64b7-4c25-9a75-716b3ee6f55b admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:59.268050 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e31fa5-696e-4de1-9f1e-e06f83c9a5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:59.268698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e31fa5-696e-4de1-9f1e-e06f83c9a5e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:59.268985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e31fa5-696e-4de1-9f1e-e06f83c9a5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:59.269262 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e31fa5-696e-4de1-9f1e-e06f83c9a5e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:59.269959 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e31fa5-696e-4de1-9f1e-e06f83c9a5e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:59.271564 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e31fa5-696e-4de1-9f1e-e06f83c9a5e0 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:59.283236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1081/2152] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:18:59.287187 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3ed0a-b9c1-4017-adda-ec630c519895 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:59.298029 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b3ed0a-b9c1-4017-adda-ec630c519895 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:59.302966 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5c726-ea29-4c20-acf3-05cd52aa8d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:59.303704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5c726-ea29-4c20-acf3-05cd52aa8d75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:59.303985 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5c726-ea29-4c20-acf3-05cd52aa8d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:59.304263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5c726-ea29-4c20-acf3-05cd52aa8d75 admin admin] PATH_INFO: `/v3/projects/51859e4f18fd421eb00ab2063f172989/users/d0237944a13e4f4ab6f5ed34ad442a2d/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:59.326075 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5c726-ea29-4c20-acf3-05cd52aa8d75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51859e4f18fd421eb00ab2063f172989, user_id=d0237944a13e4f4ab6f5ed34ad442a2d, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:59.328820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d5c726-ea29-4c20-acf3-05cd52aa8d75 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:59.344494 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1072/2153] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:59 2022] PUT /identity/v3/projects/51859e4f18fd421eb00ab2063f172989/users/d0237944a13e4f4ab6f5ed34ad442a2d/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:59.348431 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e31fa5-696e-4de1-9f1e-e06f83c9a5e0 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:59.359028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e31fa5-696e-4de1-9f1e-e06f83c9a5e0 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:59.363819 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:18:59.424234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247f4336-d5c7-420d-8493-3a2075c75401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:59.424975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f4336-d5c7-420d-8493-3a2075c75401 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:59.425259 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f4336-d5c7-420d-8493-3a2075c75401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:59.425678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247f4336-d5c7-420d-8493-3a2075c75401 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:59.426359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f4336-d5c7-420d-8493-3a2075c75401 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:59.428165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247f4336-d5c7-420d-8493-3a2075c75401 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:59.439230 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1082/2154] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:18:59.443611 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5c726-ea29-4c20-acf3-05cd52aa8d75 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:59.454217 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5c726-ea29-4c20-acf3-05cd52aa8d75 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:59.459039 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2df44-d970-44a8-a9c4-27bf298acbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:59.459949 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2df44-d970-44a8-a9c4-27bf298acbda admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:59.460225 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2df44-d970-44a8-a9c4-27bf298acbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:59.460492 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f2df44-d970-44a8-a9c4-27bf298acbda admin admin] PATH_INFO: `/v3/projects/51859e4f18fd421eb00ab2063f172989/users/d0237944a13e4f4ab6f5ed34ad442a2d/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:59.463015 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2df44-d970-44a8-a9c4-27bf298acbda admin admin] RBAC: Authorizing `identity:create_grant(project_id=51859e4f18fd421eb00ab2063f172989, user_id=d0237944a13e4f4ab6f5ed34ad442a2d, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:59.465694 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f2df44-d970-44a8-a9c4-27bf298acbda admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:59.480445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1073/2155] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:18:59 2022] PUT /identity/v3/projects/51859e4f18fd421eb00ab2063f172989/users/d0237944a13e4f4ab6f5ed34ad442a2d/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:18:59.487268 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1d553-9441-4c5c-a34e-7648a9e9a167 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:59.487552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1d553-9441-4c5c-a34e-7648a9e9a167 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:59.487826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1d553-9441-4c5c-a34e-7648a9e9a167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:59.515900 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-63b1d553-9441-4c5c-a34e-7648a9e9a167 None None] MFA Rules not processed for user `d0237944a13e4f4ab6f5ed34ad442a2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:18:59.517120 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-63b1d553-9441-4c5c-a34e-7648a9e9a167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:59.583556 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1083/2156] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:18:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:18:59.595605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2df44-d970-44a8-a9c4-27bf298acbda admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:59.597404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-00f2df44-d970-44a8-a9c4-27bf298acbda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:59.656516 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f2df44-d970-44a8-a9c4-27bf298acbda admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:59.660677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01235899-8667-4040-a78d-cd86752a5ec9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:59.661224 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01235899-8667-4040-a78d-cd86752a5ec9 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:59.661510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01235899-8667-4040-a78d-cd86752a5ec9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:59.661781 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01235899-8667-4040-a78d-cd86752a5ec9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:59.671387 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01235899-8667-4040-a78d-cd86752a5ec9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:59.672839 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01235899-8667-4040-a78d-cd86752a5ec9 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:59.682658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1074/2157] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:18:59.728756 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1d553-9441-4c5c-a34e-7648a9e9a167 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:18:59.730134 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-63b1d553-9441-4c5c-a34e-7648a9e9a167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:18:59.742178 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1d553-9441-4c5c-a34e-7648a9e9a167 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:18:59.746582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc60de6-de88-4dab-a736-b1a0d7ceef4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:18:59.747187 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc60de6-de88-4dab-a736-b1a0d7ceef4e service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:18:59.747466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc60de6-de88-4dab-a736-b1a0d7ceef4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:18:59.747746 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc60de6-de88-4dab-a736-b1a0d7ceef4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:18:59.803482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc60de6-de88-4dab-a736-b1a0d7ceef4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:18:59.804986 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc60de6-de88-4dab-a736-b1a0d7ceef4e service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:18:59.814477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1084/2158] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:18:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:19:03.206701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01235899-8667-4040-a78d-cd86752a5ec9 service cinder] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:03.279646 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01235899-8667-4040-a78d-cd86752a5ec9 service cinder] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:03.285311 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aa694-5a48-4135-b6c1-c083fbd3b6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:03.286066 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aa694-5a48-4135-b6c1-c083fbd3b6e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:03.286374 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aa694-5a48-4135-b6c1-c083fbd3b6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:03.286714 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aa694-5a48-4135-b6c1-c083fbd3b6e3 admin admin] PATH_INFO: `/v3/users/0818797b17e1444ea729d868f00ff1f4` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:03.288770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3aa694-5a48-4135-b6c1-c083fbd3b6e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0818797b17e1444ea729d868f00ff1f4)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:03.290809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3aa694-5a48-4135-b6c1-c083fbd3b6e3 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:03.329307 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:03.343413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4a3aa694-5a48-4135-b6c1-c083fbd3b6e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0818797b17e1444ea729d868f00ff1f4'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:03.344081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4a3aa694-5a48-4135-b6c1-c083fbd3b6e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0818797b17e1444ea729d868f00ff1f4'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:03.344720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4a3aa694-5a48-4135-b6c1-c083fbd3b6e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0818797b17e1444ea729d868f00ff1f4'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:03.357221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4a3aa694-5a48-4135-b6c1-c083fbd3b6e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0818797b17e1444ea729d868f00ff1f4'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:03.364018 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4a3aa694-5a48-4135-b6c1-c083fbd3b6e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0818797b17e1444ea729d868f00ff1f4'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:03.379092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1075/2159] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:19:03 2022] DELETE /identity/v3/users/0818797b17e1444ea729d868f00ff1f4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:03.383672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e359a96-9552-4a07-be55-134abaca067e None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:03.384225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e359a96-9552-4a07-be55-134abaca067e None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:03.385151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e359a96-9552-4a07-be55-134abaca067e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:03.404537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-4e359a96-9552-4a07-be55-134abaca067e None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:03.405811 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4e359a96-9552-4a07-be55-134abaca067e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:03.431778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:03.469743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1085/2160] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:19:03.481278 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aa694-5a48-4135-b6c1-c083fbd3b6e3 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:03.482947 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-4a3aa694-5a48-4135-b6c1-c083fbd3b6e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:03.540928 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aa694-5a48-4135-b6c1-c083fbd3b6e3 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:03.544750 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c12a84-f071-4f81-ac7b-016f965525b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:03.545366 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c12a84-f071-4f81-ac7b-016f965525b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:03.545650 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c12a84-f071-4f81-ac7b-016f965525b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:03.545922 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c12a84-f071-4f81-ac7b-016f965525b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:03.555365 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c12a84-f071-4f81-ac7b-016f965525b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:03.557454 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c12a84-f071-4f81-ac7b-016f965525b7 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:03.567126 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1076/2161] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:19:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:19:03.879714 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e359a96-9552-4a07-be55-134abaca067e None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:03.881789 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4e359a96-9552-4a07-be55-134abaca067e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:03.897717 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e359a96-9552-4a07-be55-134abaca067e None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:03.907151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4820d-374e-495c-8f03-946885d4f1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:03.907743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4820d-374e-495c-8f03-946885d4f1c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:03.908048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4820d-374e-495c-8f03-946885d4f1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:03.908328 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4820d-374e-495c-8f03-946885d4f1c3 admin admin] PATH_INFO: `/v3/projects/bc0f3e996b574189a7a4359d1aa64c89` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:03.909540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4820d-374e-495c-8f03-946885d4f1c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc0f3e996b574189a7a4359d1aa64c89)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:03.911447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4820d-374e-495c-8f03-946885d4f1c3 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:03.964680 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6ea4820d-374e-495c-8f03-946885d4f1c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc0f3e996b574189a7a4359d1aa64c89', 'request_id': 'req-6ea4820d-374e-495c-8f03-946885d4f1c3'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:03.973440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6ea4820d-374e-495c-8f03-946885d4f1c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc0f3e996b574189a7a4359d1aa64c89', 'request_id': 'req-6ea4820d-374e-495c-8f03-946885d4f1c3'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:03.981170 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6ea4820d-374e-495c-8f03-946885d4f1c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc0f3e996b574189a7a4359d1aa64c89', 'request_id': 'req-6ea4820d-374e-495c-8f03-946885d4f1c3'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:03.994534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6ea4820d-374e-495c-8f03-946885d4f1c3 admin admin] The token cache is being invalidate because project bc0f3e996b574189a7a4359d1aa64c89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:19:03.994830 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6ea4820d-374e-495c-8f03-946885d4f1c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:03.995473 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6ea4820d-374e-495c-8f03-946885d4f1c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:03.998213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1086/2162] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:19:03 2022] DELETE /identity/v3/projects/bc0f3e996b574189a7a4359d1aa64c89 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:04.002629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c12a84-f071-4f81-ac7b-016f965525b7 service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:04.012759 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c12a84-f071-4f81-ac7b-016f965525b7 service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:04.083843 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad449e8-f049-4136-8f78-e17fcc85e41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:04.084891 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad449e8-f049-4136-8f78-e17fcc85e41c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:04.085175 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad449e8-f049-4136-8f78-e17fcc85e41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:04.085454 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad449e8-f049-4136-8f78-e17fcc85e41c admin admin] PATH_INFO: `/v3/users/d0237944a13e4f4ab6f5ed34ad442a2d` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:04.086720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad449e8-f049-4136-8f78-e17fcc85e41c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0237944a13e4f4ab6f5ed34ad442a2d)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:04.088740 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad449e8-f049-4136-8f78-e17fcc85e41c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:04.137921 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1ad449e8-f049-4136-8f78-e17fcc85e41c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0237944a13e4f4ab6f5ed34ad442a2d'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:04.138861 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1ad449e8-f049-4136-8f78-e17fcc85e41c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0237944a13e4f4ab6f5ed34ad442a2d'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:04.139496 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1ad449e8-f049-4136-8f78-e17fcc85e41c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0237944a13e4f4ab6f5ed34ad442a2d'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:04.147072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:04.163594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1ad449e8-f049-4136-8f78-e17fcc85e41c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0237944a13e4f4ab6f5ed34ad442a2d'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:04.172082 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1ad449e8-f049-4136-8f78-e17fcc85e41c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0237944a13e4f4ab6f5ed34ad442a2d'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:04.193569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1077/2163] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:19:04 2022] DELETE /identity/v3/users/d0237944a13e4f4ab6f5ed34ad442a2d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:04.467508 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4820d-374e-495c-8f03-946885d4f1c3 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:04.469062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-6ea4820d-374e-495c-8f03-946885d4f1c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:04.495701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad449e8-f049-4136-8f78-e17fcc85e41c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:04.498226 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-1ad449e8-f049-4136-8f78-e17fcc85e41c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:04.549252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4820d-374e-495c-8f03-946885d4f1c3 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:04.581472 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad449e8-f049-4136-8f78-e17fcc85e41c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:04.586651 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c275d-602e-472a-9ca1-505af8b44016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:04.587415 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c275d-602e-472a-9ca1-505af8b44016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:04.587700 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c275d-602e-472a-9ca1-505af8b44016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:04.587996 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c275d-602e-472a-9ca1-505af8b44016 admin admin] PATH_INFO: `/v3/projects/51859e4f18fd421eb00ab2063f172989` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:04.589194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c275d-602e-472a-9ca1-505af8b44016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51859e4f18fd421eb00ab2063f172989)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:04.591124 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54c275d-602e-472a-9ca1-505af8b44016 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:04.604604 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f811a3e-f43f-48da-aca0-e245e6f30e49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:04.605256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f811a3e-f43f-48da-aca0-e245e6f30e49 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:04.605539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f811a3e-f43f-48da-aca0-e245e6f30e49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:04.605812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f811a3e-f43f-48da-aca0-e245e6f30e49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:04.607649 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-2f811a3e-f43f-48da-aca0-e245e6f30e49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:04.631637 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:04.642353 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d54c275d-602e-472a-9ca1-505af8b44016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51859e4f18fd421eb00ab2063f172989', 'request_id': 'req-d54c275d-602e-472a-9ca1-505af8b44016'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:04.653795 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d54c275d-602e-472a-9ca1-505af8b44016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51859e4f18fd421eb00ab2063f172989', 'request_id': 'req-d54c275d-602e-472a-9ca1-505af8b44016'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:04.661772 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d54c275d-602e-472a-9ca1-505af8b44016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51859e4f18fd421eb00ab2063f172989', 'request_id': 'req-d54c275d-602e-472a-9ca1-505af8b44016'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:04.673596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d54c275d-602e-472a-9ca1-505af8b44016 admin admin] The token cache is being invalidate because project 51859e4f18fd421eb00ab2063f172989 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:19:04.673924 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d54c275d-602e-472a-9ca1-505af8b44016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:04.674468 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d54c275d-602e-472a-9ca1-505af8b44016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:04.675971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1078/2164] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:19:04 2022] DELETE /identity/v3/projects/51859e4f18fd421eb00ab2063f172989 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:04.680769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7690c-cb10-4636-a450-fc1f7ef29533 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:04.681003 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f811a3e-f43f-48da-aca0-e245e6f30e49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:04.681794 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7690c-cb10-4636-a450-fc1f7ef29533 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:04.682171 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7690c-cb10-4636-a450-fc1f7ef29533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:04.682956 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f811a3e-f43f-48da-aca0-e245e6f30e49 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:04.692826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1087/2165] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:19:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:19:04.701825 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-dfc7690c-cb10-4636-a450-fc1f7ef29533 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:04.703046 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-dfc7690c-cb10-4636-a450-fc1f7ef29533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:04.773129 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1079/2166] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:19:04.778609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff9145-dc47-45ad-a1e8-84134695443c None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:04.779023 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff9145-dc47-45ad-a1e8-84134695443c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:04.779151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff9145-dc47-45ad-a1e8-84134695443c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:04.798794 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-00ff9145-dc47-45ad-a1e8-84134695443c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:04.800281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-00ff9145-dc47-45ad-a1e8-84134695443c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:04.808149 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1088/2167] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:04.812381 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7690c-cb10-4636-a450-fc1f7ef29533 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:04.817743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:04.825337 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7690c-cb10-4636-a450-fc1f7ef29533 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:04.831203 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0749047-fa79-4645-99d6-4c36cfcc2357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:04.831949 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0749047-fa79-4645-99d6-4c36cfcc2357 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:04.832333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0749047-fa79-4645-99d6-4c36cfcc2357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:04.832745 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0749047-fa79-4645-99d6-4c36cfcc2357 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:04.833584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0749047-fa79-4645-99d6-4c36cfcc2357 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:19:04.833991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0749047-fa79-4645-99d6-4c36cfcc2357 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:04.835961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0749047-fa79-4645-99d6-4c36cfcc2357 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:04.853974 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1080/2168] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:19:04.855425 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff9145-dc47-45ad-a1e8-84134695443c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:04.865730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff9145-dc47-45ad-a1e8-84134695443c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:04.870850 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4c4f0-cea5-41c7-8284-0bf0de258c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:04.871466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4c4f0-cea5-41c7-8284-0bf0de258c2f admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:04.871746 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4c4f0-cea5-41c7-8284-0bf0de258c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:04.872019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4c4f0-cea5-41c7-8284-0bf0de258c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:04.872800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4c4f0-cea5-41c7-8284-0bf0de258c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:04.874703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4c4f0-cea5-41c7-8284-0bf0de258c2f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:04.903717 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1089/2169] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:19:04 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:19:04.908003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0749047-fa79-4645-99d6-4c36cfcc2357 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:04.919571 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0749047-fa79-4645-99d6-4c36cfcc2357 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:04.978595 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7642f-40ef-4949-8389-7547d1bf6d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:04.979221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7642f-40ef-4949-8389-7547d1bf6d07 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:04.979512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7642f-40ef-4949-8389-7547d1bf6d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:04.979793 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7642f-40ef-4949-8389-7547d1bf6d07 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:04.980590 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7642f-40ef-4949-8389-7547d1bf6d07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:04.982766 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7642f-40ef-4949-8389-7547d1bf6d07 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:05.005368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1081/2170] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:19:04 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:19:05.009622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4c4f0-cea5-41c7-8284-0bf0de258c2f admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:05.020147 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4c4f0-cea5-41c7-8284-0bf0de258c2f admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:05.025565 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bf6de-ea73-452f-ba32-309de7744046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:05.026184 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bf6de-ea73-452f-ba32-309de7744046 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:05.026517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bf6de-ea73-452f-ba32-309de7744046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:05.026800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815bf6de-ea73-452f-ba32-309de7744046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:05.027522 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815bf6de-ea73-452f-ba32-309de7744046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:05.029201 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815bf6de-ea73-452f-ba32-309de7744046 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:05.041400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1090/2171] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:19:05.045649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7642f-40ef-4949-8389-7547d1bf6d07 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:05.056835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7642f-40ef-4949-8389-7547d1bf6d07 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:05.061584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e1c3ea-e1d5-4b6e-9af0-91ae645a5dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:05.062213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1c3ea-e1d5-4b6e-9af0-91ae645a5dea admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:05.062495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1c3ea-e1d5-4b6e-9af0-91ae645a5dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:05.062774 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e1c3ea-e1d5-4b6e-9af0-91ae645a5dea admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:05.063471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e1c3ea-e1d5-4b6e-9af0-91ae645a5dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:05.065150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e1c3ea-e1d5-4b6e-9af0-91ae645a5dea admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:05.080263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1082/2172] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:19:05.082168 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bf6de-ea73-452f-ba32-309de7744046 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:05.095015 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815bf6de-ea73-452f-ba32-309de7744046 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:05.100066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72416de-d8dc-4de3-9616-f48426484ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:05.101118 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72416de-d8dc-4de3-9616-f48426484ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:05.101348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72416de-d8dc-4de3-9616-f48426484ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:05.101696 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72416de-d8dc-4de3-9616-f48426484ee8 admin admin] PATH_INFO: `/v3/projects/6408c1efb7d64efca45be9312a95e01a/users/68fa6ad44aa14bfdb7d275d9ff1ed7bf/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:05.120221 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72416de-d8dc-4de3-9616-f48426484ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6408c1efb7d64efca45be9312a95e01a, user_id=68fa6ad44aa14bfdb7d275d9ff1ed7bf, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:05.123178 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72416de-d8dc-4de3-9616-f48426484ee8 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:05.143840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1091/2173] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:19:05 2022] PUT /identity/v3/projects/6408c1efb7d64efca45be9312a95e01a/users/68fa6ad44aa14bfdb7d275d9ff1ed7bf/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:05.155718 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6565cd-d3cf-4e99-97de-e1be52cf65e8 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:05.156106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6565cd-d3cf-4e99-97de-e1be52cf65e8 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:05.156396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6565cd-d3cf-4e99-97de-e1be52cf65e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:05.196326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-df6565cd-d3cf-4e99-97de-e1be52cf65e8 None None] MFA Rules not processed for user `68fa6ad44aa14bfdb7d275d9ff1ed7bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:05.198471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-df6565cd-d3cf-4e99-97de-e1be52cf65e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:05.288461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1083/2174] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:05.294135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72416de-d8dc-4de3-9616-f48426484ee8 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:05.385107 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72416de-d8dc-4de3-9616-f48426484ee8 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:05.391465 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490308a9-a0e6-42c1-ad5e-52e518978bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:05.392090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490308a9-a0e6-42c1-ad5e-52e518978bc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:05.392366 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490308a9-a0e6-42c1-ad5e-52e518978bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:05.392641 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490308a9-a0e6-42c1-ad5e-52e518978bc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:05.393371 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490308a9-a0e6-42c1-ad5e-52e518978bc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:05.395437 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490308a9-a0e6-42c1-ad5e-52e518978bc2 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:05.431675 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1092/2175] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:19:05 2022] POST /identity/v3/projects => generated 393 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:19:05.437577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6565cd-d3cf-4e99-97de-e1be52cf65e8 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:05.454385 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6565cd-d3cf-4e99-97de-e1be52cf65e8 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:05.513613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:05.520560 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f1ff3-627a-4575-a4fb-d44488672310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:05.521301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f1ff3-627a-4575-a4fb-d44488672310 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:05.521581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f1ff3-627a-4575-a4fb-d44488672310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:05.521959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999f1ff3-627a-4575-a4fb-d44488672310 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:05.522520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f1ff3-627a-4575-a4fb-d44488672310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:05.525299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999f1ff3-627a-4575-a4fb-d44488672310 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:05.548854 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1084/2176] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:19:05 2022] POST /identity/v3/users => generated 303 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:19:05.553265 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490308a9-a0e6-42c1-ad5e-52e518978bc2 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:05.565180 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490308a9-a0e6-42c1-ad5e-52e518978bc2 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:05.570064 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e89cf5-c76b-4998-b0ec-d049c0d6e52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:05.570689 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e89cf5-c76b-4998-b0ec-d049c0d6e52c admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:05.570974 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e89cf5-c76b-4998-b0ec-d049c0d6e52c admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:05.571252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e89cf5-c76b-4998-b0ec-d049c0d6e52c admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:05.571991 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e89cf5-c76b-4998-b0ec-d049c0d6e52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:05.573696 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e89cf5-c76b-4998-b0ec-d049c0d6e52c admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:05.586072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1093/2177] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:19:05.590693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f1ff3-627a-4575-a4fb-d44488672310 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:05.601306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999f1ff3-627a-4575-a4fb-d44488672310 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:05.607240 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be917393-7f91-4aec-b288-ac8bbb5d8b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:05.607920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be917393-7f91-4aec-b288-ac8bbb5d8b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:05.608203 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be917393-7f91-4aec-b288-ac8bbb5d8b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:05.608477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be917393-7f91-4aec-b288-ac8bbb5d8b7f admin admin] PATH_INFO: `/v3/projects/1e1c1955d4764fd1817462179a09bf0d/users/e8bc9f1db8ae44a5a5892f274f621eba/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:05.624397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be917393-7f91-4aec-b288-ac8bbb5d8b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e1c1955d4764fd1817462179a09bf0d, user_id=e8bc9f1db8ae44a5a5892f274f621eba, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:05.627148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be917393-7f91-4aec-b288-ac8bbb5d8b7f admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:05.669328 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1085/2178] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:19:05 2022] PUT /identity/v3/projects/1e1c1955d4764fd1817462179a09bf0d/users/e8bc9f1db8ae44a5a5892f274f621eba/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:05.673228 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e89cf5-c76b-4998-b0ec-d049c0d6e52c admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:05.686252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e89cf5-c76b-4998-b0ec-d049c0d6e52c admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:05.718221 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:05.753355 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc0e99-fd7a-42ae-9079-0fa1101aec7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:05.753977 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc0e99-fd7a-42ae-9079-0fa1101aec7f admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:05.754260 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc0e99-fd7a-42ae-9079-0fa1101aec7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:05.754552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc0e99-fd7a-42ae-9079-0fa1101aec7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:05.755241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc0e99-fd7a-42ae-9079-0fa1101aec7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:05.756914 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc0e99-fd7a-42ae-9079-0fa1101aec7f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:05.768262 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1094/2179] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:19:05.772122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be917393-7f91-4aec-b288-ac8bbb5d8b7f admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:05.782677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be917393-7f91-4aec-b288-ac8bbb5d8b7f admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:05.787820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2dff0-83ba-4d6a-8e62-d19f8a1ffcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:05.788557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2dff0-83ba-4d6a-8e62-d19f8a1ffcc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:05.788943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2dff0-83ba-4d6a-8e62-d19f8a1ffcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:05.789278 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2dff0-83ba-4d6a-8e62-d19f8a1ffcc4 admin admin] PATH_INFO: `/v3/projects/1e1c1955d4764fd1817462179a09bf0d/users/e8bc9f1db8ae44a5a5892f274f621eba/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:05.791390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e2dff0-83ba-4d6a-8e62-d19f8a1ffcc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e1c1955d4764fd1817462179a09bf0d, user_id=e8bc9f1db8ae44a5a5892f274f621eba, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:05.794210 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e2dff0-83ba-4d6a-8e62-d19f8a1ffcc4 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:05.805631 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1086/2180] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:19:05 2022] PUT /identity/v3/projects/1e1c1955d4764fd1817462179a09bf0d/users/e8bc9f1db8ae44a5a5892f274f621eba/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:05.817226 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759b1a5-a86a-487f-a6d1-b20e34585569 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:05.817664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759b1a5-a86a-487f-a6d1-b20e34585569 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:05.818008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759b1a5-a86a-487f-a6d1-b20e34585569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:05.849393 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-f759b1a5-a86a-487f-a6d1-b20e34585569 None None] MFA Rules not processed for user `e8bc9f1db8ae44a5a5892f274f621eba`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:05.850811 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f759b1a5-a86a-487f-a6d1-b20e34585569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:05.937753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1095/2181] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:05 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:05.950316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2dff0-83ba-4d6a-8e62-d19f8a1ffcc4 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:05.952800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-21e2dff0-83ba-4d6a-8e62-d19f8a1ffcc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:06.037408 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e2dff0-83ba-4d6a-8e62-d19f8a1ffcc4 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:06.042477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff1616-306b-4192-b46e-f3b667172943 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:06.042659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff1616-306b-4192-b46e-f3b667172943 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:06.042805 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff1616-306b-4192-b46e-f3b667172943 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:06.043083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff1616-306b-4192-b46e-f3b667172943 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:06.052857 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff1616-306b-4192-b46e-f3b667172943 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:06.054326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff1616-306b-4192-b46e-f3b667172943 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:06.065512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1087/2182] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:19:05 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:19:06.114273 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759b1a5-a86a-487f-a6d1-b20e34585569 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:06.115680 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f759b1a5-a86a-487f-a6d1-b20e34585569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:06.128430 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759b1a5-a86a-487f-a6d1-b20e34585569 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:06.132137 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5942b2-0244-487d-8e64-5ea85cd02f2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:06.132875 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5942b2-0244-487d-8e64-5ea85cd02f2a service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:06.133219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5942b2-0244-487d-8e64-5ea85cd02f2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:06.133554 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5942b2-0244-487d-8e64-5ea85cd02f2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:06.194304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5942b2-0244-487d-8e64-5ea85cd02f2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:06.195870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5942b2-0244-487d-8e64-5ea85cd02f2a service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:06.205722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1096/2183] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:19:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:19:09.713144 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff1616-306b-4192-b46e-f3b667172943 service cinder] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:09.775721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff1616-306b-4192-b46e-f3b667172943 service cinder] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:09.781246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff597a2c-5b29-4fdc-8e01-ea142d60758c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:09.781919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff597a2c-5b29-4fdc-8e01-ea142d60758c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:09.782200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff597a2c-5b29-4fdc-8e01-ea142d60758c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:09.782477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff597a2c-5b29-4fdc-8e01-ea142d60758c admin admin] PATH_INFO: `/v3/users/68fa6ad44aa14bfdb7d275d9ff1ed7bf` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:09.790419 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff597a2c-5b29-4fdc-8e01-ea142d60758c admin admin] RBAC: Authorizing `identity:delete_user(user_id=68fa6ad44aa14bfdb7d275d9ff1ed7bf)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:09.792343 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff597a2c-5b29-4fdc-8e01-ea142d60758c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:09.847180 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ff597a2c-5b29-4fdc-8e01-ea142d60758c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68fa6ad44aa14bfdb7d275d9ff1ed7bf'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:09.847847 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ff597a2c-5b29-4fdc-8e01-ea142d60758c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68fa6ad44aa14bfdb7d275d9ff1ed7bf'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:09.848522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ff597a2c-5b29-4fdc-8e01-ea142d60758c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68fa6ad44aa14bfdb7d275d9ff1ed7bf'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:09.854134 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:09.862348 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ff597a2c-5b29-4fdc-8e01-ea142d60758c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68fa6ad44aa14bfdb7d275d9ff1ed7bf'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:09.869768 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ff597a2c-5b29-4fdc-8e01-ea142d60758c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68fa6ad44aa14bfdb7d275d9ff1ed7bf'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:09.883944 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1088/2184] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:19:09 2022] DELETE /identity/v3/users/68fa6ad44aa14bfdb7d275d9ff1ed7bf => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:09.888771 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550a0a8-9f07-43b1-8fb1-21fa97f4c5a9 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:09.889242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550a0a8-9f07-43b1-8fb1-21fa97f4c5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:09.889345 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9550a0a8-9f07-43b1-8fb1-21fa97f4c5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:09.909032 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-9550a0a8-9f07-43b1-8fb1-21fa97f4c5a9 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:09.910255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9550a0a8-9f07-43b1-8fb1-21fa97f4c5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:09.955941 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:09.972909 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1097/2185] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:09.982612 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff597a2c-5b29-4fdc-8e01-ea142d60758c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:09.984010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ff597a2c-5b29-4fdc-8e01-ea142d60758c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:10.044994 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff597a2c-5b29-4fdc-8e01-ea142d60758c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:10.049373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171ce81-f99c-4e4f-870f-5fa2809f43ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:10.050000 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171ce81-f99c-4e4f-870f-5fa2809f43ed service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:10.050284 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171ce81-f99c-4e4f-870f-5fa2809f43ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:10.050559 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171ce81-f99c-4e4f-870f-5fa2809f43ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:10.060701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171ce81-f99c-4e4f-870f-5fa2809f43ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:10.062137 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171ce81-f99c-4e4f-870f-5fa2809f43ed service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:10.071522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1089/2186] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:19:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:19:10.451048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550a0a8-9f07-43b1-8fb1-21fa97f4c5a9 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:10.453072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9550a0a8-9f07-43b1-8fb1-21fa97f4c5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:10.479464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9550a0a8-9f07-43b1-8fb1-21fa97f4c5a9 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:10.502090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88183443-6b2e-4f85-a261-e39cc90b78e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:10.503732 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88183443-6b2e-4f85-a261-e39cc90b78e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:10.504844 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88183443-6b2e-4f85-a261-e39cc90b78e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:10.506085 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88183443-6b2e-4f85-a261-e39cc90b78e5 admin admin] PATH_INFO: `/v3/projects/6408c1efb7d64efca45be9312a95e01a` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:10.509508 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88183443-6b2e-4f85-a261-e39cc90b78e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6408c1efb7d64efca45be9312a95e01a)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:10.512605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88183443-6b2e-4f85-a261-e39cc90b78e5 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:10.612563 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-88183443-6b2e-4f85-a261-e39cc90b78e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6408c1efb7d64efca45be9312a95e01a', 'request_id': 'req-88183443-6b2e-4f85-a261-e39cc90b78e5'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:10.621905 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-88183443-6b2e-4f85-a261-e39cc90b78e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6408c1efb7d64efca45be9312a95e01a', 'request_id': 'req-88183443-6b2e-4f85-a261-e39cc90b78e5'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:10.628154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-88183443-6b2e-4f85-a261-e39cc90b78e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6408c1efb7d64efca45be9312a95e01a', 'request_id': 'req-88183443-6b2e-4f85-a261-e39cc90b78e5'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:10.641915 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-88183443-6b2e-4f85-a261-e39cc90b78e5 admin admin] The token cache is being invalidate because project 6408c1efb7d64efca45be9312a95e01a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:19:10.642208 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-88183443-6b2e-4f85-a261-e39cc90b78e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:10.644834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-88183443-6b2e-4f85-a261-e39cc90b78e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:10.649732 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1098/2187] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:19:10 2022] DELETE /identity/v3/projects/6408c1efb7d64efca45be9312a95e01a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:10.654322 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171ce81-f99c-4e4f-870f-5fa2809f43ed service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:10.666832 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171ce81-f99c-4e4f-870f-5fa2809f43ed service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:10.763514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f80191-4a33-4db5-a19e-1e11571d0cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:10.764167 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f80191-4a33-4db5-a19e-1e11571d0cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:10.764442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f80191-4a33-4db5-a19e-1e11571d0cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:10.764750 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f80191-4a33-4db5-a19e-1e11571d0cf9 admin admin] PATH_INFO: `/v3/users/e8bc9f1db8ae44a5a5892f274f621eba` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:10.766000 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f80191-4a33-4db5-a19e-1e11571d0cf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8bc9f1db8ae44a5a5892f274f621eba)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:10.767872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f80191-4a33-4db5-a19e-1e11571d0cf9 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:10.825566 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a2f80191-4a33-4db5-a19e-1e11571d0cf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8bc9f1db8ae44a5a5892f274f621eba'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:10.826217 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a2f80191-4a33-4db5-a19e-1e11571d0cf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8bc9f1db8ae44a5a5892f274f621eba'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:10.826856 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a2f80191-4a33-4db5-a19e-1e11571d0cf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8bc9f1db8ae44a5a5892f274f621eba'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:10.840689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a2f80191-4a33-4db5-a19e-1e11571d0cf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8bc9f1db8ae44a5a5892f274f621eba'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:10.849514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a2f80191-4a33-4db5-a19e-1e11571d0cf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8bc9f1db8ae44a5a5892f274f621eba'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:10.852713 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:10.872355 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1090/2188] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:19:10 2022] DELETE /identity/v3/users/e8bc9f1db8ae44a5a5892f274f621eba => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:11.172343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88183443-6b2e-4f85-a261-e39cc90b78e5 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:11.172343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-88183443-6b2e-4f85-a261-e39cc90b78e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:11.186045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88183443-6b2e-4f85-a261-e39cc90b78e5 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:11.245056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19fd52-5446-4fdf-acc9-cdda9e935573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:11.245698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19fd52-5446-4fdf-acc9-cdda9e935573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:11.245977 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19fd52-5446-4fdf-acc9-cdda9e935573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:11.246254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19fd52-5446-4fdf-acc9-cdda9e935573 admin admin] PATH_INFO: `/v3/projects/1e1c1955d4764fd1817462179a09bf0d` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:11.247412 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19fd52-5446-4fdf-acc9-cdda9e935573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e1c1955d4764fd1817462179a09bf0d)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:11.249462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19fd52-5446-4fdf-acc9-cdda9e935573 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:11.295875 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-af19fd52-5446-4fdf-acc9-cdda9e935573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e1c1955d4764fd1817462179a09bf0d', 'request_id': 'req-af19fd52-5446-4fdf-acc9-cdda9e935573'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:11.303501 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-af19fd52-5446-4fdf-acc9-cdda9e935573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e1c1955d4764fd1817462179a09bf0d', 'request_id': 'req-af19fd52-5446-4fdf-acc9-cdda9e935573'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:11.309123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-af19fd52-5446-4fdf-acc9-cdda9e935573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e1c1955d4764fd1817462179a09bf0d', 'request_id': 'req-af19fd52-5446-4fdf-acc9-cdda9e935573'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:11.320051 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-af19fd52-5446-4fdf-acc9-cdda9e935573 admin admin] The token cache is being invalidate because project 1e1c1955d4764fd1817462179a09bf0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:19:11.320333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-af19fd52-5446-4fdf-acc9-cdda9e935573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:11.321039 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-af19fd52-5446-4fdf-acc9-cdda9e935573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:11.323054 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1099/2189] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:19:11 2022] DELETE /identity/v3/projects/1e1c1955d4764fd1817462179a09bf0d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:11.327918 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50ee0c-df9f-48d6-af9a-0bc735ece1ca None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:11.328203 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50ee0c-df9f-48d6-af9a-0bc735ece1ca None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:11.328480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c50ee0c-df9f-48d6-af9a-0bc735ece1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:11.348844 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-5c50ee0c-df9f-48d6-af9a-0bc735ece1ca None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:11.350043 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-5c50ee0c-df9f-48d6-af9a-0bc735ece1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:11.413390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1091/2190] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:11.421281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847b70a-3c0c-4d0b-bbfc-41ae0656c825 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:11.422027 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847b70a-3c0c-4d0b-bbfc-41ae0656c825 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:11.422027 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847b70a-3c0c-4d0b-bbfc-41ae0656c825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:11.429534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:11.444652 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-f847b70a-3c0c-4d0b-bbfc-41ae0656c825 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:11.445933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f847b70a-3c0c-4d0b-bbfc-41ae0656c825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:11.454449 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1100/2191] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:11.460227 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50ee0c-df9f-48d6-af9a-0bc735ece1ca None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:11.475455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c50ee0c-df9f-48d6-af9a-0bc735ece1ca None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:11.481323 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7263d4e-ef59-4a7c-a111-757571d74fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:11.482017 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7263d4e-ef59-4a7c-a111-757571d74fa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:11.482418 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7263d4e-ef59-4a7c-a111-757571d74fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:11.482727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7263d4e-ef59-4a7c-a111-757571d74fa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:11.483638 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7263d4e-ef59-4a7c-a111-757571d74fa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:19:11.484025 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7263d4e-ef59-4a7c-a111-757571d74fa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:11.486172 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7263d4e-ef59-4a7c-a111-757571d74fa8 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:11.503720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1092/2192] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:19:11.504902 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847b70a-3c0c-4d0b-bbfc-41ae0656c825 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:11.514840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847b70a-3c0c-4d0b-bbfc-41ae0656c825 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:11.519478 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260496c-3b93-4449-93ce-1b43049a8e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:11.520151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260496c-3b93-4449-93ce-1b43049a8e7e admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:11.520351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260496c-3b93-4449-93ce-1b43049a8e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:11.520641 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7260496c-3b93-4449-93ce-1b43049a8e7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:11.521303 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260496c-3b93-4449-93ce-1b43049a8e7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:11.523178 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7260496c-3b93-4449-93ce-1b43049a8e7e admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:11.550538 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1101/2193] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:19:11 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:19:11.555004 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7263d4e-ef59-4a7c-a111-757571d74fa8 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:11.568306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7263d4e-ef59-4a7c-a111-757571d74fa8 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:11.631946 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e204f88-1fee-47b6-acfc-9a2c928d36d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:11.632567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e204f88-1fee-47b6-acfc-9a2c928d36d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:11.632893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e204f88-1fee-47b6-acfc-9a2c928d36d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:11.633164 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e204f88-1fee-47b6-acfc-9a2c928d36d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:11.633813 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e204f88-1fee-47b6-acfc-9a2c928d36d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:11.636329 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e204f88-1fee-47b6-acfc-9a2c928d36d8 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:11.659472 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1093/2194] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:19:11 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:19:11.663968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260496c-3b93-4449-93ce-1b43049a8e7e admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:11.674730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7260496c-3b93-4449-93ce-1b43049a8e7e admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:11.679494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70409fd-d4f9-4ec4-ad97-b546bc9ec210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:11.680129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70409fd-d4f9-4ec4-ad97-b546bc9ec210 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:11.680410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70409fd-d4f9-4ec4-ad97-b546bc9ec210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:11.680716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70409fd-d4f9-4ec4-ad97-b546bc9ec210 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:11.681912 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70409fd-d4f9-4ec4-ad97-b546bc9ec210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:11.683531 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70409fd-d4f9-4ec4-ad97-b546bc9ec210 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:11.693787 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1102/2195] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:19:11.698584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e204f88-1fee-47b6-acfc-9a2c928d36d8 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:11.708744 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e204f88-1fee-47b6-acfc-9a2c928d36d8 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:11.714010 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20ce91d-045d-4ecb-bc41-8cdd6c110639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:11.714634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ce91d-045d-4ecb-bc41-8cdd6c110639 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:11.715337 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ce91d-045d-4ecb-bc41-8cdd6c110639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:11.715337 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20ce91d-045d-4ecb-bc41-8cdd6c110639 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:11.715871 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ce91d-045d-4ecb-bc41-8cdd6c110639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:11.717518 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20ce91d-045d-4ecb-bc41-8cdd6c110639 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:11.730196 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1094/2196] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:19:11.734860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70409fd-d4f9-4ec4-ad97-b546bc9ec210 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:11.746315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70409fd-d4f9-4ec4-ad97-b546bc9ec210 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:11.751123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645e235-9917-42bf-a600-219184a79c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:11.751897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645e235-9917-42bf-a600-219184a79c13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:11.752272 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645e235-9917-42bf-a600-219184a79c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:11.752661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3645e235-9917-42bf-a600-219184a79c13 admin admin] PATH_INFO: `/v3/projects/b9eaf54498e2461293dfdac1c10d80b8/users/8224236aeab64179b91dcd5cb5c0e58c/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:11.787259 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645e235-9917-42bf-a600-219184a79c13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9eaf54498e2461293dfdac1c10d80b8, user_id=8224236aeab64179b91dcd5cb5c0e58c, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:11.792087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3645e235-9917-42bf-a600-219184a79c13 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:11.809743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1103/2197] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:19:11 2022] PUT /identity/v3/projects/b9eaf54498e2461293dfdac1c10d80b8/users/8224236aeab64179b91dcd5cb5c0e58c/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:11.819872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e633c-62a8-45f8-af1d-4b046d4c5ae2 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:11.820407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e633c-62a8-45f8-af1d-4b046d4c5ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:11.820407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0e633c-62a8-45f8-af1d-4b046d4c5ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:11.883950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-2a0e633c-62a8-45f8-af1d-4b046d4c5ae2 None None] MFA Rules not processed for user `8224236aeab64179b91dcd5cb5c0e58c`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:11.885805 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2a0e633c-62a8-45f8-af1d-4b046d4c5ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:11.970953 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:11.991117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1095/2198] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:12.002451 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645e235-9917-42bf-a600-219184a79c13 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:12.004185 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-3645e235-9917-42bf-a600-219184a79c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:12.082951 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3645e235-9917-42bf-a600-219184a79c13 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:12.088859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486ad5d0-8990-49b8-8248-3d34c64cb94a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ff31d52ecf4c7d93a65bbfca7285c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:12.089407 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ad5d0-8990-49b8-8248-3d34c64cb94a service glance] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:12.089692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ad5d0-8990-49b8-8248-3d34c64cb94a service glance] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:12.089955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486ad5d0-8990-49b8-8248-3d34c64cb94a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:12.098677 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ad5d0-8990-49b8-8248-3d34c64cb94a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:12.100094 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486ad5d0-8990-49b8-8248-3d34c64cb94a service glance] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:12.109254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1104/2199] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:19:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:19:14.609660 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba557fd-c255-4ac0-83d2-6ef84730efd6 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:14.610225 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba557fd-c255-4ac0-83d2-6ef84730efd6 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:14.610415 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba557fd-c255-4ac0-83d2-6ef84730efd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:14.630520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-9ba557fd-c255-4ac0-83d2-6ef84730efd6 None None] MFA Rules not processed for user `033f13b2943945d3abc4ff852b68d698`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:14.632128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9ba557fd-c255-4ac0-83d2-6ef84730efd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:14.695548 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1096/2200] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:22.105200 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=61,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:24.688945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=30 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:24.862646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091abf86-054d-4a4d-8cb0-a9d39d38efd3 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:24.863147 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091abf86-054d-4a4d-8cb0-a9d39d38efd3 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:24.863244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091abf86-054d-4a4d-8cb0-a9d39d38efd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:24.892772 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-091abf86-054d-4a4d-8cb0-a9d39d38efd3 None None] MFA Rules not processed for user `e54e713b52e84393b018807f8153922e`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:24.894327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-091abf86-054d-4a4d-8cb0-a9d39d38efd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:24.976235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1105/2201] 10.222.0.45 () {58 vars in 1009 bytes} [Thu Aug 4 02:19:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:25.028568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba557fd-c255-4ac0-83d2-6ef84730efd6 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:25.030670 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9ba557fd-c255-4ac0-83d2-6ef84730efd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:25.140747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba557fd-c255-4ac0-83d2-6ef84730efd6 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:25.148340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63b514-9bb6-450e-8968-397027840ecb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '155a7276efab426cb34175683c475f33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:25.149105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63b514-9bb6-450e-8968-397027840ecb service swift] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:25.149459 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63b514-9bb6-450e-8968-397027840ecb service swift] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:25.151035 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63b514-9bb6-450e-8968-397027840ecb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:25.159670 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63b514-9bb6-450e-8968-397027840ecb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:25.161439 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63b514-9bb6-450e-8968-397027840ecb service swift] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:25.172213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1097/2202] 10.222.0.45 () {60 vars in 1384 bytes} [Thu Aug 4 02:19:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 02:19:26.844679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec650f-12c4-420f-86f6-be9cd8aed159 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:26.845300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec650f-12c4-420f-86f6-be9cd8aed159 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:26.845300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec650f-12c4-420f-86f6-be9cd8aed159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:26.867455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-0aec650f-12c4-420f-86f6-be9cd8aed159 None None] MFA Rules not processed for user `76703dc82dc644ef8d32f7e259ddea6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:26.869244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-0aec650f-12c4-420f-86f6-be9cd8aed159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:27.005767 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1106/2203] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:35.168477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=32 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:36.985882 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=50 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:44.704358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736b1d4-acce-4286-84d8-0946b368502a None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:44.704926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736b1d4-acce-4286-84d8-0946b368502a None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:44.705169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736b1d4-acce-4286-84d8-0946b368502a None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:44.706965 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1098/2204] 10.222.0.45 () {54 vars in 921 bytes} [Thu Aug 4 02:19:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:19:44.714364 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b166ba8-f14f-49ae-a5c7-b66f0ff6ca0d None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:44.714364 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b166ba8-f14f-49ae-a5c7-b66f0ff6ca0d None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:44.714364 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b166ba8-f14f-49ae-a5c7-b66f0ff6ca0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:44.758115 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-8b166ba8-f14f-49ae-a5c7-b66f0ff6ca0d None None] MFA Rules not processed for user `c571406d2a3f40dab035a918399d6f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:44.759423 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8b166ba8-f14f-49ae-a5c7-b66f0ff6ca0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:44.875661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1107/2205] 10.222.0.45 () {58 vars in 1018 bytes} [Thu Aug 4 02:19:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:44.893449 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736b1d4-acce-4286-84d8-0946b368502a None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:44.905819 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2736b1d4-acce-4286-84d8-0946b368502a None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:44.909683 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711df589-7f9a-49af-bcad-3c7e46dd9a15 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ff31d52ecf4c7d93a65bbfca7285c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:44.910296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711df589-7f9a-49af-bcad-3c7e46dd9a15 service glance] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:44.910572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711df589-7f9a-49af-bcad-3c7e46dd9a15 service glance] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:44.910844 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711df589-7f9a-49af-bcad-3c7e46dd9a15 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:44.923320 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711df589-7f9a-49af-bcad-3c7e46dd9a15 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:44.926782 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711df589-7f9a-49af-bcad-3c7e46dd9a15 service glance] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:44.939043 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1099/2206] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:19:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:19:45.010572 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8270de-994a-4925-915a-502fe20bb194 None None] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:45.010872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8270de-994a-4925-915a-502fe20bb194 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:45.011139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8270de-994a-4925-915a-502fe20bb194 None None] PATH_INFO: `/` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:45.012687 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1108/2207] 10.222.0.45 () {54 vars in 921 bytes} [Thu Aug 4 02:19:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:19:45.017727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8811fd5-eb49-486e-af72-352e7a9facf2 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:45.018851 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8811fd5-eb49-486e-af72-352e7a9facf2 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:45.019129 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8811fd5-eb49-486e-af72-352e7a9facf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:45.039038 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-f8811fd5-eb49-486e-af72-352e7a9facf2 None None] MFA Rules not processed for user `6c2da427acb24c8cb6d667c6347858c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:45.040235 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-f8811fd5-eb49-486e-af72-352e7a9facf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:45.097285 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1100/2208] 10.222.0.45 () {58 vars in 1018 bytes} [Thu Aug 4 02:19:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:45.109410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8270de-994a-4925-915a-502fe20bb194 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:45.111164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4e8270de-994a-4925-915a-502fe20bb194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:45.125783 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8270de-994a-4925-915a-502fe20bb194 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:45.129574 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887273d7-326d-4f0e-8bc3-74047dad4e6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:45.130176 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887273d7-326d-4f0e-8bc3-74047dad4e6d service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:45.130455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887273d7-326d-4f0e-8bc3-74047dad4e6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:45.130724 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887273d7-326d-4f0e-8bc3-74047dad4e6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:45.138482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887273d7-326d-4f0e-8bc3-74047dad4e6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:45.139936 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887273d7-326d-4f0e-8bc3-74047dad4e6d service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:45.148282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1109/2209] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:19:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:19:45.235198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47347b03-eeaa-4980-8219-c379a4631540 None None] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:45.235481 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47347b03-eeaa-4980-8219-c379a4631540 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:45.235752 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47347b03-eeaa-4980-8219-c379a4631540 None None] PATH_INFO: `/` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:45.237174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1101/2210] 10.222.0.45 () {54 vars in 921 bytes} [Thu Aug 4 02:19:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 04 02:19:45.242361 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672f537-4d2b-4998-8073-1f241f62b9b9 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:45.242652 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672f537-4d2b-4998-8073-1f241f62b9b9 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:45.242927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672f537-4d2b-4998-8073-1f241f62b9b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:45.263276 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c672f537-4d2b-4998-8073-1f241f62b9b9 None None] MFA Rules not processed for user `c571406d2a3f40dab035a918399d6f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:45.264487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c672f537-4d2b-4998-8073-1f241f62b9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:45.273776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1110/2211] 10.222.0.45 () {58 vars in 1018 bytes} [Thu Aug 4 02:19:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:45.284379 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47347b03-eeaa-4980-8219-c379a4631540 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:45.286113 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-47347b03-eeaa-4980-8219-c379a4631540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:45.299528 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47347b03-eeaa-4980-8219-c379a4631540 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:45.303185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6957942-b39e-4f9d-bad3-0d4eb505ba26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:45.303792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6957942-b39e-4f9d-bad3-0d4eb505ba26 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:45.304072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6957942-b39e-4f9d-bad3-0d4eb505ba26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:45.304346 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6957942-b39e-4f9d-bad3-0d4eb505ba26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:45.315596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6957942-b39e-4f9d-bad3-0d4eb505ba26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:45.317365 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6957942-b39e-4f9d-bad3-0d4eb505ba26 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:45.329636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1102/2212] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:19:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:19:48.273576 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672f537-4d2b-4998-8073-1f241f62b9b9 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:48.363177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672f537-4d2b-4998-8073-1f241f62b9b9 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:48.370828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632ceaf-4090-4904-b434-d7ab89d24ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:48.371481 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632ceaf-4090-4904-b434-d7ab89d24ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:48.371759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632ceaf-4090-4904-b434-d7ab89d24ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:48.372034 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632ceaf-4090-4904-b434-d7ab89d24ed6 admin admin] PATH_INFO: `/v3/users/8224236aeab64179b91dcd5cb5c0e58c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:48.373251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632ceaf-4090-4904-b434-d7ab89d24ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8224236aeab64179b91dcd5cb5c0e58c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:48.375461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632ceaf-4090-4904-b434-d7ab89d24ed6 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:48.441198 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-9632ceaf-4090-4904-b434-d7ab89d24ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8224236aeab64179b91dcd5cb5c0e58c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:48.449206 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-9632ceaf-4090-4904-b434-d7ab89d24ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8224236aeab64179b91dcd5cb5c0e58c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:48.449939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-9632ceaf-4090-4904-b434-d7ab89d24ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8224236aeab64179b91dcd5cb5c0e58c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:48.462637 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-9632ceaf-4090-4904-b434-d7ab89d24ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8224236aeab64179b91dcd5cb5c0e58c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:48.463368 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-9632ceaf-4090-4904-b434-d7ab89d24ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8224236aeab64179b91dcd5cb5c0e58c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:48.465808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:48.485248 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1111/2213] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:19:48 2022] DELETE /identity/v3/users/8224236aeab64179b91dcd5cb5c0e58c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:48.487139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034276a-4927-4e74-8852-cd6531fd78a0 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:48.487619 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034276a-4927-4e74-8852-cd6531fd78a0 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:48.487932 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034276a-4927-4e74-8852-cd6531fd78a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:48.509373 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-3034276a-4927-4e74-8852-cd6531fd78a0 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:48.511264 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-3034276a-4927-4e74-8852-cd6531fd78a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:48.562863 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56ad88-0f33-4db0-a5f3-8871759f74b4 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:48.563245 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56ad88-0f33-4db0-a5f3-8871759f74b4 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:48.563587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56ad88-0f33-4db0-a5f3-8871759f74b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:48.584182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-7e56ad88-0f33-4db0-a5f3-8871759f74b4 None None] MFA Rules not processed for user `e54e713b52e84393b018807f8153922e`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:48.585428 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7e56ad88-0f33-4db0-a5f3-8871759f74b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:48.586322 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1103/2214] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:48.594949 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034276a-4927-4e74-8852-cd6531fd78a0 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:48.597487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-3034276a-4927-4e74-8852-cd6531fd78a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:48.655532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1112/2215] 10.222.0.45 () {58 vars in 1009 bytes} [Thu Aug 4 02:19:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:19:48.679438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56ad88-0f33-4db0-a5f3-8871759f74b4 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:48.681040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7e56ad88-0f33-4db0-a5f3-8871759f74b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:48.682256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034276a-4927-4e74-8852-cd6531fd78a0 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:48.686615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acad8dd-f727-47b2-9ec2-69faed8c7c02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:48.688015 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acad8dd-f727-47b2-9ec2-69faed8c7c02 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:48.688367 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acad8dd-f727-47b2-9ec2-69faed8c7c02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:48.688870 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acad8dd-f727-47b2-9ec2-69faed8c7c02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:48.700751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acad8dd-f727-47b2-9ec2-69faed8c7c02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:48.702786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acad8dd-f727-47b2-9ec2-69faed8c7c02 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:48.716017 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1104/2216] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:19:48.767459 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56ad88-0f33-4db0-a5f3-8871759f74b4 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:48.774466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae0a86-82b7-46e4-90e5-cdb009ea30e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '155a7276efab426cb34175683c475f33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:48.775250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae0a86-82b7-46e4-90e5-cdb009ea30e1 service swift] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:48.775741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae0a86-82b7-46e4-90e5-cdb009ea30e1 service swift] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:48.775960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae0a86-82b7-46e4-90e5-cdb009ea30e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:48.788010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae0a86-82b7-46e4-90e5-cdb009ea30e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:48.789802 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae0a86-82b7-46e4-90e5-cdb009ea30e1 service swift] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:48.799844 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1113/2217] 10.222.0.45 () {60 vars in 1384 bytes} [Thu Aug 4 02:19:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 02:19:49.060072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acad8dd-f727-47b2-9ec2-69faed8c7c02 service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:49.061513 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-4acad8dd-f727-47b2-9ec2-69faed8c7c02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:49.093256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acad8dd-f727-47b2-9ec2-69faed8c7c02 service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:49.098662 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70949668-9b73-4314-8551-2062b4b6c244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:49.099362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70949668-9b73-4314-8551-2062b4b6c244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:49.099705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70949668-9b73-4314-8551-2062b4b6c244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:49.100045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70949668-9b73-4314-8551-2062b4b6c244 admin admin] PATH_INFO: `/v3/projects/b9eaf54498e2461293dfdac1c10d80b8` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:49.103446 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70949668-9b73-4314-8551-2062b4b6c244 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9eaf54498e2461293dfdac1c10d80b8)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:49.106269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70949668-9b73-4314-8551-2062b4b6c244 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:49.162504 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:49.206848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-70949668-9b73-4314-8551-2062b4b6c244 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9eaf54498e2461293dfdac1c10d80b8', 'request_id': 'req-70949668-9b73-4314-8551-2062b4b6c244'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:49.221881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-70949668-9b73-4314-8551-2062b4b6c244 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9eaf54498e2461293dfdac1c10d80b8', 'request_id': 'req-70949668-9b73-4314-8551-2062b4b6c244'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:49.237279 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-70949668-9b73-4314-8551-2062b4b6c244 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9eaf54498e2461293dfdac1c10d80b8', 'request_id': 'req-70949668-9b73-4314-8551-2062b4b6c244'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:49.250091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-70949668-9b73-4314-8551-2062b4b6c244 admin admin] The token cache is being invalidate because project b9eaf54498e2461293dfdac1c10d80b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:19:49.250525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-70949668-9b73-4314-8551-2062b4b6c244 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:49.251295 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-70949668-9b73-4314-8551-2062b4b6c244 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:49.256292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1105/2218] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:19:49 2022] DELETE /identity/v3/projects/b9eaf54498e2461293dfdac1c10d80b8 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:49.258162 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d6d01-c5b4-462a-b6e6-29faa368ab0f None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:49.258605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d6d01-c5b4-462a-b6e6-29faa368ab0f None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:49.259031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d6d01-c5b4-462a-b6e6-29faa368ab0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:49.285508 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-da6d6d01-c5b4-462a-b6e6-29faa368ab0f None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:49.286885 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-da6d6d01-c5b4-462a-b6e6-29faa368ab0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:49.434837 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1114/2219] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:49.436868 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93583f10-d512-45f1-9eab-7cc22f6308dc None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:49.437161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93583f10-d512-45f1-9eab-7cc22f6308dc None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:49.440201 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93583f10-d512-45f1-9eab-7cc22f6308dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:49.486360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-93583f10-d512-45f1-9eab-7cc22f6308dc None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:49.487520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-93583f10-d512-45f1-9eab-7cc22f6308dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:49.508371 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1106/2220] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:49.509830 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d6d01-c5b4-462a-b6e6-29faa368ab0f None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:49.524594 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d6d01-c5b4-462a-b6e6-29faa368ab0f None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:49.541323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf745e-0ddc-4cac-9f34-e37db8d0dddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:49.542146 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf745e-0ddc-4cac-9f34-e37db8d0dddd admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:49.542530 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf745e-0ddc-4cac-9f34-e37db8d0dddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:49.542906 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf745e-0ddc-4cac-9f34-e37db8d0dddd admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:49.543736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf745e-0ddc-4cac-9f34-e37db8d0dddd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:19:49.544145 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf745e-0ddc-4cac-9f34-e37db8d0dddd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:49.545910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf745e-0ddc-4cac-9f34-e37db8d0dddd admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:49.580452 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93583f10-d512-45f1-9eab-7cc22f6308dc None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:49.584939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1115/2221] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:19:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:19:49.596018 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93583f10-d512-45f1-9eab-7cc22f6308dc None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:49.608700 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f88ada-a955-4e24-87fb-106d8f36f982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:49.610309 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f88ada-a955-4e24-87fb-106d8f36f982 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:49.610791 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f88ada-a955-4e24-87fb-106d8f36f982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:49.612172 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f88ada-a955-4e24-87fb-106d8f36f982 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:49.614755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f88ada-a955-4e24-87fb-106d8f36f982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:49.625543 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f88ada-a955-4e24-87fb-106d8f36f982 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:49.699355 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1107/2222] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:19:49 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:19:49.702881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf745e-0ddc-4cac-9f34-e37db8d0dddd admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:49.731534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf745e-0ddc-4cac-9f34-e37db8d0dddd admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:49.741880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:49.833177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126e906-a4fc-435a-9c04-ba5666e8a4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:49.833856 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126e906-a4fc-435a-9c04-ba5666e8a4d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:49.834267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126e906-a4fc-435a-9c04-ba5666e8a4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:49.834543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126e906-a4fc-435a-9c04-ba5666e8a4d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:49.835212 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126e906-a4fc-435a-9c04-ba5666e8a4d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:49.837462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126e906-a4fc-435a-9c04-ba5666e8a4d5 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:49.866177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1116/2223] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:19:49 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:19:49.867391 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f88ada-a955-4e24-87fb-106d8f36f982 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:49.881912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f88ada-a955-4e24-87fb-106d8f36f982 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:49.889827 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabeaad6-6ef6-43bc-81dd-7c1eded7b9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:49.890457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabeaad6-6ef6-43bc-81dd-7c1eded7b9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:49.890736 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabeaad6-6ef6-43bc-81dd-7c1eded7b9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:49.891005 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabeaad6-6ef6-43bc-81dd-7c1eded7b9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:49.891722 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabeaad6-6ef6-43bc-81dd-7c1eded7b9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:49.893386 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabeaad6-6ef6-43bc-81dd-7c1eded7b9f2 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:49.905441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1108/2224] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:19:49.910196 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126e906-a4fc-435a-9c04-ba5666e8a4d5 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:49.921479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126e906-a4fc-435a-9c04-ba5666e8a4d5 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:49.926271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0b3408-35d6-432c-8c8e-5f91fd995b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:49.927007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b3408-35d6-432c-8c8e-5f91fd995b3c admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:49.927401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b3408-35d6-432c-8c8e-5f91fd995b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:49.927770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0b3408-35d6-432c-8c8e-5f91fd995b3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:49.928739 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b3408-35d6-432c-8c8e-5f91fd995b3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:49.930599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0b3408-35d6-432c-8c8e-5f91fd995b3c admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:49.945428 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1117/2225] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:19:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:19:49.950261 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabeaad6-6ef6-43bc-81dd-7c1eded7b9f2 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:49.962336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabeaad6-6ef6-43bc-81dd-7c1eded7b9f2 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:49.969096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a080-1e33-431a-af81-d83fb89bd0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:49.970152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a080-1e33-431a-af81-d83fb89bd0bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:49.970653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a080-1e33-431a-af81-d83fb89bd0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:49.971140 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a080-1e33-431a-af81-d83fb89bd0bc admin admin] PATH_INFO: `/v3/projects/bb54269523f545c59692387904ed7b0f/users/5eb1f82917484d409a40c068d85a32a7/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:49.989676 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a080-1e33-431a-af81-d83fb89bd0bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb54269523f545c59692387904ed7b0f, user_id=5eb1f82917484d409a40c068d85a32a7, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:49.992374 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a080-1e33-431a-af81-d83fb89bd0bc admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:50.009989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1109/2226] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:19:49 2022] PUT /identity/v3/projects/bb54269523f545c59692387904ed7b0f/users/5eb1f82917484d409a40c068d85a32a7/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:50.021461 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b7208-da01-47e6-8246-3da569e79807 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:50.021838 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b7208-da01-47e6-8246-3da569e79807 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:50.022113 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7b7208-da01-47e6-8246-3da569e79807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:50.055134 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-7f7b7208-da01-47e6-8246-3da569e79807 None None] MFA Rules not processed for user `5eb1f82917484d409a40c068d85a32a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:50.056365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7f7b7208-da01-47e6-8246-3da569e79807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:50.155086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1118/2227] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:50.163024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a080-1e33-431a-af81-d83fb89bd0bc admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:50.164898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-84e8a080-1e33-431a-af81-d83fb89bd0bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:50.247891 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a080-1e33-431a-af81-d83fb89bd0bc admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:50.251699 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ae38d-3715-40b4-83c8-37a0a9c29397 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:50.252326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ae38d-3715-40b4-83c8-37a0a9c29397 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:50.252606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ae38d-3715-40b4-83c8-37a0a9c29397 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:50.252904 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ae38d-3715-40b4-83c8-37a0a9c29397 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:50.279332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ae38d-3715-40b4-83c8-37a0a9c29397 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:50.280950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ae38d-3715-40b4-83c8-37a0a9c29397 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:50.291111 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1110/2228] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:19:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:19:50.321557 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b7208-da01-47e6-8246-3da569e79807 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:50.402512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7b7208-da01-47e6-8246-3da569e79807 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:50.407378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8dce7-2e5c-4643-9385-0abc8900b6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:50.408026 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8dce7-2e5c-4643-9385-0abc8900b6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:50.408484 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8dce7-2e5c-4643-9385-0abc8900b6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:50.408799 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8dce7-2e5c-4643-9385-0abc8900b6f0 admin admin] PATH_INFO: `/v3/users/5eb1f82917484d409a40c068d85a32a7` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:50.410030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8dce7-2e5c-4643-9385-0abc8900b6f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eb1f82917484d409a40c068d85a32a7)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:50.412043 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8dce7-2e5c-4643-9385-0abc8900b6f0 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:50.440477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:50.481646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d3c8dce7-2e5c-4643-9385-0abc8900b6f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eb1f82917484d409a40c068d85a32a7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:50.492116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d3c8dce7-2e5c-4643-9385-0abc8900b6f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eb1f82917484d409a40c068d85a32a7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:50.493001 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d3c8dce7-2e5c-4643-9385-0abc8900b6f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eb1f82917484d409a40c068d85a32a7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:50.517886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d3c8dce7-2e5c-4643-9385-0abc8900b6f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eb1f82917484d409a40c068d85a32a7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:50.519051 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d3c8dce7-2e5c-4643-9385-0abc8900b6f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eb1f82917484d409a40c068d85a32a7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:50.539591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1119/2229] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:19:50 2022] DELETE /identity/v3/users/5eb1f82917484d409a40c068d85a32a7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:50.544352 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258171a3-8843-4889-876b-428958b59eed None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:50.544919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258171a3-8843-4889-876b-428958b59eed None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:50.545178 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258171a3-8843-4889-876b-428958b59eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:50.574347 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-258171a3-8843-4889-876b-428958b59eed None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:50.575821 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-258171a3-8843-4889-876b-428958b59eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:50.630687 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:50.653121 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1111/2230] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:50.665657 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8dce7-2e5c-4643-9385-0abc8900b6f0 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:50.667807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d3c8dce7-2e5c-4643-9385-0abc8900b6f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:50.748427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8dce7-2e5c-4643-9385-0abc8900b6f0 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:50.755186 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf66356-eec6-4ba6-a5d2-de13300643d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:50.755829 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf66356-eec6-4ba6-a5d2-de13300643d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:50.756112 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf66356-eec6-4ba6-a5d2-de13300643d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:50.756388 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf66356-eec6-4ba6-a5d2-de13300643d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:50.767081 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf66356-eec6-4ba6-a5d2-de13300643d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:50.768676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf66356-eec6-4ba6-a5d2-de13300643d8 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:50.794946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1120/2231] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:19:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:19:51.238933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258171a3-8843-4889-876b-428958b59eed None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:51.240383 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-258171a3-8843-4889-876b-428958b59eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:51.254055 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258171a3-8843-4889-876b-428958b59eed None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:51.259860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a145ee5-d934-4ff5-975e-15007f6c3636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:51.260533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a145ee5-d934-4ff5-975e-15007f6c3636 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:51.261500 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a145ee5-d934-4ff5-975e-15007f6c3636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:51.261785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a145ee5-d934-4ff5-975e-15007f6c3636 admin admin] PATH_INFO: `/v3/projects/bb54269523f545c59692387904ed7b0f` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:51.262985 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a145ee5-d934-4ff5-975e-15007f6c3636 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb54269523f545c59692387904ed7b0f)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:51.265023 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a145ee5-d934-4ff5-975e-15007f6c3636 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:51.329598 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4a145ee5-d934-4ff5-975e-15007f6c3636 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb54269523f545c59692387904ed7b0f', 'request_id': 'req-4a145ee5-d934-4ff5-975e-15007f6c3636'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:51.338504 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4a145ee5-d934-4ff5-975e-15007f6c3636 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb54269523f545c59692387904ed7b0f', 'request_id': 'req-4a145ee5-d934-4ff5-975e-15007f6c3636'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:51.345468 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4a145ee5-d934-4ff5-975e-15007f6c3636 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb54269523f545c59692387904ed7b0f', 'request_id': 'req-4a145ee5-d934-4ff5-975e-15007f6c3636'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:51.357685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4a145ee5-d934-4ff5-975e-15007f6c3636 admin admin] The token cache is being invalidate because project bb54269523f545c59692387904ed7b0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:19:51.357972 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4a145ee5-d934-4ff5-975e-15007f6c3636 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:51.358643 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4a145ee5-d934-4ff5-975e-15007f6c3636 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:19:51.360171 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1112/2232] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:19:51 2022] DELETE /identity/v3/projects/bb54269523f545c59692387904ed7b0f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:51.365641 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec079b9-6c30-4ecb-a1d9-33166b9adb5c None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:51.366171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec079b9-6c30-4ecb-a1d9-33166b9adb5c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:51.366171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec079b9-6c30-4ecb-a1d9-33166b9adb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:51.387893 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-3ec079b9-6c30-4ecb-a1d9-33166b9adb5c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:51.389328 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-3ec079b9-6c30-4ecb-a1d9-33166b9adb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:51.453271 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1121/2233] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:19:51.461561 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17aa48-a115-4ab4-9f19-9b5a08b543e9 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:51.461841 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17aa48-a115-4ab4-9f19-9b5a08b543e9 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:51.462115 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17aa48-a115-4ab4-9f19-9b5a08b543e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:51.484145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-0b17aa48-a115-4ab4-9f19-9b5a08b543e9 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:51.485475 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-0b17aa48-a115-4ab4-9f19-9b5a08b543e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:51.495492 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1113/2234] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:51.499593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec079b9-6c30-4ecb-a1d9-33166b9adb5c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:51.509798 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec079b9-6c30-4ecb-a1d9-33166b9adb5c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:51.526416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebad044-083f-4162-8194-21df678eb1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:51.528457 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebad044-083f-4162-8194-21df678eb1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:51.528850 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebad044-083f-4162-8194-21df678eb1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:51.529223 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebad044-083f-4162-8194-21df678eb1d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:51.530050 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebad044-083f-4162-8194-21df678eb1d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:19:51.530509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebad044-083f-4162-8194-21df678eb1d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:51.532684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebad044-083f-4162-8194-21df678eb1d9 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:51.552061 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1122/2235] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:19:51.553709 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17aa48-a115-4ab4-9f19-9b5a08b543e9 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:51.565081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17aa48-a115-4ab4-9f19-9b5a08b543e9 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:51.570543 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2dd35-1484-4710-bf0d-6bc7aa01fde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:51.571631 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2dd35-1484-4710-bf0d-6bc7aa01fde0 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:51.572150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2dd35-1484-4710-bf0d-6bc7aa01fde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:51.572696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2dd35-1484-4710-bf0d-6bc7aa01fde0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:51.574078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2dd35-1484-4710-bf0d-6bc7aa01fde0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:51.577567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2dd35-1484-4710-bf0d-6bc7aa01fde0 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:51.614275 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1114/2236] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:19:51 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:19:51.618429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebad044-083f-4162-8194-21df678eb1d9 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:51.630289 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebad044-083f-4162-8194-21df678eb1d9 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:51.683057 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:19:51.719910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16898c-bfc8-41ba-8830-cc815336afe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:51.720542 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16898c-bfc8-41ba-8830-cc815336afe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:51.720856 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16898c-bfc8-41ba-8830-cc815336afe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:51.721134 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16898c-bfc8-41ba-8830-cc815336afe8 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:51.721991 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf16898c-bfc8-41ba-8830-cc815336afe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:51.723964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf16898c-bfc8-41ba-8830-cc815336afe8 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:51.750948 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1123/2237] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:19:51 2022] POST /identity/v3/users => generated 288 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:19:51.752163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2dd35-1484-4710-bf0d-6bc7aa01fde0 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:51.763754 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2dd35-1484-4710-bf0d-6bc7aa01fde0 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:51.768496 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e351f2b-1fa6-4d3f-86a7-5ef902559d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:51.769270 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e351f2b-1fa6-4d3f-86a7-5ef902559d2b admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:51.769550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e351f2b-1fa6-4d3f-86a7-5ef902559d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:51.770051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e351f2b-1fa6-4d3f-86a7-5ef902559d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:51.770757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e351f2b-1fa6-4d3f-86a7-5ef902559d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:51.772544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e351f2b-1fa6-4d3f-86a7-5ef902559d2b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:51.784873 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1115/2238] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:19:51.788546 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16898c-bfc8-41ba-8830-cc815336afe8 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:51.799763 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16898c-bfc8-41ba-8830-cc815336afe8 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:51.805380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5dac4b-872f-453f-9fc9-6b9b834b26b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:51.806165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5dac4b-872f-453f-9fc9-6b9b834b26b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:51.806515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5dac4b-872f-453f-9fc9-6b9b834b26b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:51.806862 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5dac4b-872f-453f-9fc9-6b9b834b26b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:51.808227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5dac4b-872f-453f-9fc9-6b9b834b26b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:51.809938 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5dac4b-872f-453f-9fc9-6b9b834b26b1 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:51.825758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1124/2239] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:19:51.826907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e351f2b-1fa6-4d3f-86a7-5ef902559d2b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:51.840396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e351f2b-1fa6-4d3f-86a7-5ef902559d2b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:51.845150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201700d-8e58-4247-a4a0-32ec6db57942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:51.845828 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201700d-8e58-4247-a4a0-32ec6db57942 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:51.846109 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201700d-8e58-4247-a4a0-32ec6db57942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:51.846383 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201700d-8e58-4247-a4a0-32ec6db57942 admin admin] PATH_INFO: `/v3/projects/7febb232a825434092e8c7b660f299a4/users/a613d788888a46d29cd807aa44f0de88/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:51.863364 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201700d-8e58-4247-a4a0-32ec6db57942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7febb232a825434092e8c7b660f299a4, user_id=a613d788888a46d29cd807aa44f0de88, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:51.866183 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201700d-8e58-4247-a4a0-32ec6db57942 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:51.882991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1116/2240] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:19:51 2022] PUT /identity/v3/projects/7febb232a825434092e8c7b660f299a4/users/a613d788888a46d29cd807aa44f0de88/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:19:51.893825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498ceb1-eff4-474d-97d0-ded14e6a25f0 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:51.894111 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498ceb1-eff4-474d-97d0-ded14e6a25f0 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:51.894390 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498ceb1-eff4-474d-97d0-ded14e6a25f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:51.925801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-8498ceb1-eff4-474d-97d0-ded14e6a25f0 None None] MFA Rules not processed for user `a613d788888a46d29cd807aa44f0de88`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:19:51.927152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8498ceb1-eff4-474d-97d0-ded14e6a25f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:52.025934 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1125/2241] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:19:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:19:52.033839 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201700d-8e58-4247-a4a0-32ec6db57942 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:19:52.035564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9201700d-8e58-4247-a4a0-32ec6db57942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:19:52.098139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201700d-8e58-4247-a4a0-32ec6db57942 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:19:52.101799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964e3a7-81b7-4912-8064-323ca24b8124 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:19:52.102431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964e3a7-81b7-4912-8064-323ca24b8124 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:19:52.102714 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964e3a7-81b7-4912-8064-323ca24b8124 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:19:52.102991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964e3a7-81b7-4912-8064-323ca24b8124 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:19:52.112685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964e3a7-81b7-4912-8064-323ca24b8124 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:19:52.114205 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8964e3a7-81b7-4912-8064-323ca24b8124 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:19:52.124416 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1117/2242] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:19:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:20:02.014127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=52,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:20:02.120812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:20:02.658611 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498ceb1-eff4-474d-97d0-ded14e6a25f0 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:20:02.720880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498ceb1-eff4-474d-97d0-ded14e6a25f0 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:20:02.725395 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc1009-00fc-4a12-8a58-d96a52d90602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:20:02.726023 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc1009-00fc-4a12-8a58-d96a52d90602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:20:02.726306 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc1009-00fc-4a12-8a58-d96a52d90602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:20:02.726582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc1009-00fc-4a12-8a58-d96a52d90602 admin admin] PATH_INFO: `/v3/users/a613d788888a46d29cd807aa44f0de88` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:20:02.727774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc1009-00fc-4a12-8a58-d96a52d90602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a613d788888a46d29cd807aa44f0de88)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:20:02.730406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc1009-00fc-4a12-8a58-d96a52d90602 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:20:02.775199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-67fc1009-00fc-4a12-8a58-d96a52d90602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a613d788888a46d29cd807aa44f0de88'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:20:02.782435 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-67fc1009-00fc-4a12-8a58-d96a52d90602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a613d788888a46d29cd807aa44f0de88'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:20:02.783093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-67fc1009-00fc-4a12-8a58-d96a52d90602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a613d788888a46d29cd807aa44f0de88'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:20:02.797093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-67fc1009-00fc-4a12-8a58-d96a52d90602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a613d788888a46d29cd807aa44f0de88'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:20:02.798458 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-67fc1009-00fc-4a12-8a58-d96a52d90602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a613d788888a46d29cd807aa44f0de88'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:20:02.811682 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1126/2243] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:20:02 2022] DELETE /identity/v3/users/a613d788888a46d29cd807aa44f0de88 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:20:02.816226 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f79f47-3c94-4ddd-8127-73de1723eabf None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:20:02.816518 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f79f47-3c94-4ddd-8127-73de1723eabf None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:20:02.816839 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f79f47-3c94-4ddd-8127-73de1723eabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:20:02.835744 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-c5f79f47-3c94-4ddd-8127-73de1723eabf None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:20:02.837238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-c5f79f47-3c94-4ddd-8127-73de1723eabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:20:02.898645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1118/2244] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:20:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:20:02.908753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc1009-00fc-4a12-8a58-d96a52d90602 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:20:02.910179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-67fc1009-00fc-4a12-8a58-d96a52d90602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:20:02.975113 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc1009-00fc-4a12-8a58-d96a52d90602 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:20:02.978941 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff50d6a-9a26-4449-b8b6-c37eeb6e8b25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:20:02.979635 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff50d6a-9a26-4449-b8b6-c37eeb6e8b25 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:20:02.979987 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff50d6a-9a26-4449-b8b6-c37eeb6e8b25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:20:02.980330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff50d6a-9a26-4449-b8b6-c37eeb6e8b25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:20:02.989721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff50d6a-9a26-4449-b8b6-c37eeb6e8b25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:20:02.991251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff50d6a-9a26-4449-b8b6-c37eeb6e8b25 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:20:03.002605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1127/2245] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:20:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:20:03.508388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f79f47-3c94-4ddd-8127-73de1723eabf None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:20:03.509884 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-c5f79f47-3c94-4ddd-8127-73de1723eabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:20:03.524080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f79f47-3c94-4ddd-8127-73de1723eabf None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:20:03.529059 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f62090d-30b4-41a1-89e3-9982a50c6b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:20:03.529740 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62090d-30b4-41a1-89e3-9982a50c6b7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:20:03.530019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62090d-30b4-41a1-89e3-9982a50c6b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:20:03.530292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f62090d-30b4-41a1-89e3-9982a50c6b7f admin admin] PATH_INFO: `/v3/projects/7febb232a825434092e8c7b660f299a4` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:20:03.531499 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f62090d-30b4-41a1-89e3-9982a50c6b7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7febb232a825434092e8c7b660f299a4)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:20:03.533549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f62090d-30b4-41a1-89e3-9982a50c6b7f admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:20:03.581431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4f62090d-30b4-41a1-89e3-9982a50c6b7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7febb232a825434092e8c7b660f299a4', 'request_id': 'req-4f62090d-30b4-41a1-89e3-9982a50c6b7f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:20:03.588913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4f62090d-30b4-41a1-89e3-9982a50c6b7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7febb232a825434092e8c7b660f299a4', 'request_id': 'req-4f62090d-30b4-41a1-89e3-9982a50c6b7f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:20:03.594467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4f62090d-30b4-41a1-89e3-9982a50c6b7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7febb232a825434092e8c7b660f299a4', 'request_id': 'req-4f62090d-30b4-41a1-89e3-9982a50c6b7f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:20:03.605830 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4f62090d-30b4-41a1-89e3-9982a50c6b7f admin admin] The token cache is being invalidate because project 7febb232a825434092e8c7b660f299a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:20:03.606115 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4f62090d-30b4-41a1-89e3-9982a50c6b7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:20:03.606774 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4f62090d-30b4-41a1-89e3-9982a50c6b7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:20:03.608287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1119/2246] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:20:03 2022] DELETE /identity/v3/projects/7febb232a825434092e8c7b660f299a4 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:20:07.053800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff50d6a-9a26-4449-b8b6-c37eeb6e8b25 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:20:07.055268 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-0ff50d6a-9a26-4449-b8b6-c37eeb6e8b25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:20:07.080307 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:20:07.125756 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff50d6a-9a26-4449-b8b6-c37eeb6e8b25 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:20:07.183180 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be6233-95ed-4c89-9d57-f92edf16e67f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796772cf491047c8b0a609dae9402307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:20:07.183787 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be6233-95ed-4c89-9d57-f92edf16e67f service placement] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:20:07.184064 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be6233-95ed-4c89-9d57-f92edf16e67f service placement] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:20:07.184332 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be6233-95ed-4c89-9d57-f92edf16e67f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:20:07.186178 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-38be6233-95ed-4c89-9d57-f92edf16e67f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:20:07.197137 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be6233-95ed-4c89-9d57-f92edf16e67f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:20:07.198842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be6233-95ed-4c89-9d57-f92edf16e67f service placement] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:20:07.208460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1128/2247] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:20:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:20:13.310137 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac19ed-dde0-4fee-8f7e-1c9b9597de04 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:20:13.310626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac19ed-dde0-4fee-8f7e-1c9b9597de04 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:20:13.310738 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac19ed-dde0-4fee-8f7e-1c9b9597de04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:20:13.330282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-10ac19ed-dde0-4fee-8f7e-1c9b9597de04 None None] MFA Rules not processed for user `e54e713b52e84393b018807f8153922e`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:20:13.331467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-10ac19ed-dde0-4fee-8f7e-1c9b9597de04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:20:13.387350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1120/2248] 10.222.0.45 () {58 vars in 967 bytes} [Thu Aug 4 02:20:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:20:13.402033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be6233-95ed-4c89-9d57-f92edf16e67f service placement] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:20:13.403444 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-38be6233-95ed-4c89-9d57-f92edf16e67f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:20:13.472490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be6233-95ed-4c89-9d57-f92edf16e67f service placement] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:20:13.477454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92356a2-6ee2-40b9-ba8f-c4b5b78c6090 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '155a7276efab426cb34175683c475f33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:20:13.478064 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92356a2-6ee2-40b9-ba8f-c4b5b78c6090 service swift] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:20:13.478341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92356a2-6ee2-40b9-ba8f-c4b5b78c6090 service swift] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:20:13.478610 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92356a2-6ee2-40b9-ba8f-c4b5b78c6090 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:20:13.488678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92356a2-6ee2-40b9-ba8f-c4b5b78c6090 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:20:13.490259 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92356a2-6ee2-40b9-ba8f-c4b5b78c6090 service swift] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:20:13.498590 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1129/2249] 10.222.0.45 () {60 vars in 1384 bytes} [Thu Aug 4 02:20:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 02:20:23.379673 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:20:23.495212 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=80 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:20:47.859101 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f75aa-1f69-453b-8b16-afb0f2343ece None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:20:47.859608 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f75aa-1f69-453b-8b16-afb0f2343ece None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:20:47.859717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709f75aa-1f69-453b-8b16-afb0f2343ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:20:47.878766 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-709f75aa-1f69-453b-8b16-afb0f2343ece None None] MFA Rules not processed for user `76703dc82dc644ef8d32f7e259ddea6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:20:47.879978 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-709f75aa-1f69-453b-8b16-afb0f2343ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:20:47.937608 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1121/2250] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:20:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:20:57.302314 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1615d84-0f86-4be1-b668-c7bc633656ec None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:20:57.302839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1615d84-0f86-4be1-b668-c7bc633656ec None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:20:57.302839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1615d84-0f86-4be1-b668-c7bc633656ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:20:57.321358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-d1615d84-0f86-4be1-b668-c7bc633656ec None None] MFA Rules not processed for user `033f13b2943945d3abc4ff852b68d698`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:20:57.322530 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d1615d84-0f86-4be1-b668-c7bc633656ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:20:57.376254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1130/2251] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:20:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:20:57.932457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=25 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:21:05.313838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629e89c-3e53-490e-8945-ba7cd2c416c7 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:21:05.314585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629e89c-3e53-490e-8945-ba7cd2c416c7 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:21:05.314585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629e89c-3e53-490e-8945-ba7cd2c416c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:21:05.333938 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-f629e89c-3e53-490e-8945-ba7cd2c416c7 None None] MFA Rules not processed for user `df5d3de1afa945c3b0873050eca06c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:21:05.335152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-f629e89c-3e53-490e-8945-ba7cd2c416c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:21:05.392792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1122/2252] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:21:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:21:07.369648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=25 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:21:15.385218 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=25 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:21:34.158673 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d167f-1051-4d75-a682-37c88a2a4242 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:21:34.159191 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d167f-1051-4d75-a682-37c88a2a4242 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:21:34.159191 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d167f-1051-4d75-a682-37c88a2a4242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:21:34.178632 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-5a5d167f-1051-4d75-a682-37c88a2a4242 None None] MFA Rules not processed for user `26e73db77d93496883b66572d6bf35f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:21:34.179829 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-5a5d167f-1051-4d75-a682-37c88a2a4242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:21:34.237754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1131/2253] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:21:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:21:44.230213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=25 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:21:53.913421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629e89c-3e53-490e-8945-ba7cd2c416c7 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:21:53.923410 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-f629e89c-3e53-490e-8945-ba7cd2c416c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:21:53.988787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629e89c-3e53-490e-8945-ba7cd2c416c7 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:21:53.992731 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3438f3-1297-40d2-97bc-f96582607428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:21:53.993409 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3438f3-1297-40d2-97bc-f96582607428 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:21:53.993685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3438f3-1297-40d2-97bc-f96582607428 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:21:53.993955 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3438f3-1297-40d2-97bc-f96582607428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:21:53.995748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-1f3438f3-1297-40d2-97bc-f96582607428 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:21:54.066665 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3438f3-1297-40d2-97bc-f96582607428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:21:54.068005 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3438f3-1297-40d2-97bc-f96582607428 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:21:54.078865 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1123/2254] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:21:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:21:54.881333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d167f-1051-4d75-a682-37c88a2a4242 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:21:54.891225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d167f-1051-4d75-a682-37c88a2a4242 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:21:54.895287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfe5cc-ab42-4bdc-bb8d-6f8298b7a7fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:21:54.895890 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfe5cc-ab42-4bdc-bb8d-6f8298b7a7fa service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:21:54.896166 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfe5cc-ab42-4bdc-bb8d-6f8298b7a7fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:21:54.896433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dfe5cc-ab42-4bdc-bb8d-6f8298b7a7fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:21:54.898175 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-84dfe5cc-ab42-4bdc-bb8d-6f8298b7a7fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:21:54.912127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dfe5cc-ab42-4bdc-bb8d-6f8298b7a7fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:21:54.913518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dfe5cc-ab42-4bdc-bb8d-6f8298b7a7fa service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:21:54.925351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1132/2255] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:21:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:22:01.035619 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312a1d5-98d2-48f0-be7a-64060c14be48 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:01.036185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312a1d5-98d2-48f0-be7a-64060c14be48 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:01.036913 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312a1d5-98d2-48f0-be7a-64060c14be48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:01.056369 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-e312a1d5-98d2-48f0-be7a-64060c14be48 None None] MFA Rules not processed for user `76703dc82dc644ef8d32f7e259ddea6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:01.057596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e312a1d5-98d2-48f0-be7a-64060c14be48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:01.064524 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1124/2256] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:01.907571 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfe5cc-ab42-4bdc-bb8d-6f8298b7a7fa service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:01.917056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dfe5cc-ab42-4bdc-bb8d-6f8298b7a7fa service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:01.920840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991a90f-3bd1-4536-8df6-34ec6e354b76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:01.921457 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991a90f-3bd1-4536-8df6-34ec6e354b76 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:01.921760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991a90f-3bd1-4536-8df6-34ec6e354b76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:01.922009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991a90f-3bd1-4536-8df6-34ec6e354b76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:01.923925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8991a90f-3bd1-4536-8df6-34ec6e354b76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:01.936220 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991a90f-3bd1-4536-8df6-34ec6e354b76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:01.937931 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991a90f-3bd1-4536-8df6-34ec6e354b76 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:01.947474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1133/2257] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:22:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:22:03.390931 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312a1d5-98d2-48f0-be7a-64060c14be48 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:03.392396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e312a1d5-98d2-48f0-be7a-64060c14be48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:03.404881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312a1d5-98d2-48f0-be7a-64060c14be48 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:03.408577 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83a763-ebbe-424b-9069-74d7b61f9ebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:03.409218 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83a763-ebbe-424b-9069-74d7b61f9ebc service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:03.409495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83a763-ebbe-424b-9069-74d7b61f9ebc service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:03.409763 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83a763-ebbe-424b-9069-74d7b61f9ebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:03.411970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9f83a763-ebbe-424b-9069-74d7b61f9ebc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:03.478062 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83a763-ebbe-424b-9069-74d7b61f9ebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:03.479557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83a763-ebbe-424b-9069-74d7b61f9ebc service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:03.491067 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1125/2258] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:22:07.257740 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991a90f-3bd1-4536-8df6-34ec6e354b76 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:07.259193 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8991a90f-3bd1-4536-8df6-34ec6e354b76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:07.274918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991a90f-3bd1-4536-8df6-34ec6e354b76 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:07.279987 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a176d-c47c-459e-a5fb-228c958a0a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:07.280782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a176d-c47c-459e-a5fb-228c958a0a3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:07.281167 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a176d-c47c-459e-a5fb-228c958a0a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:07.281547 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a176d-c47c-459e-a5fb-228c958a0a3a admin admin] PATH_INFO: `/v3/users/033f13b2943945d3abc4ff852b68d698` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:07.282783 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a176d-c47c-459e-a5fb-228c958a0a3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=033f13b2943945d3abc4ff852b68d698)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:07.284748 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a176d-c47c-459e-a5fb-228c958a0a3a admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:07.335071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1a8a176d-c47c-459e-a5fb-228c958a0a3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '033f13b2943945d3abc4ff852b68d698'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:07.342397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1a8a176d-c47c-459e-a5fb-228c958a0a3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '033f13b2943945d3abc4ff852b68d698'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:07.343156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1a8a176d-c47c-459e-a5fb-228c958a0a3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '033f13b2943945d3abc4ff852b68d698'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:07.355628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1a8a176d-c47c-459e-a5fb-228c958a0a3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '033f13b2943945d3abc4ff852b68d698'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:07.356422 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1a8a176d-c47c-459e-a5fb-228c958a0a3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '033f13b2943945d3abc4ff852b68d698'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:07.371834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1134/2259] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:22:07 2022] DELETE /identity/v3/users/033f13b2943945d3abc4ff852b68d698 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:07.562954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83a763-ebbe-424b-9069-74d7b61f9ebc service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:07.574944 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83a763-ebbe-424b-9069-74d7b61f9ebc service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:07.607406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:07.637986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a3e1c-b47c-4332-9763-ff578f9f3c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:07.638663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a3e1c-b47c-4332-9763-ff578f9f3c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:07.638943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a3e1c-b47c-4332-9763-ff578f9f3c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:07.639217 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a3e1c-b47c-4332-9763-ff578f9f3c97 admin admin] PATH_INFO: `/v3/projects/c61d4ea2b52c497caeb298fe8677488e` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:07.640381 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a3e1c-b47c-4332-9763-ff578f9f3c97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c61d4ea2b52c497caeb298fe8677488e)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:07.642312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a3e1c-b47c-4332-9763-ff578f9f3c97 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:07.695106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-433a3e1c-b47c-4332-9763-ff578f9f3c97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c61d4ea2b52c497caeb298fe8677488e', 'request_id': 'req-433a3e1c-b47c-4332-9763-ff578f9f3c97'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:07.702595 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-433a3e1c-b47c-4332-9763-ff578f9f3c97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c61d4ea2b52c497caeb298fe8677488e', 'request_id': 'req-433a3e1c-b47c-4332-9763-ff578f9f3c97'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:07.709018 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-433a3e1c-b47c-4332-9763-ff578f9f3c97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c61d4ea2b52c497caeb298fe8677488e', 'request_id': 'req-433a3e1c-b47c-4332-9763-ff578f9f3c97'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:07.721715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-433a3e1c-b47c-4332-9763-ff578f9f3c97 admin admin] The token cache is being invalidate because project c61d4ea2b52c497caeb298fe8677488e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:22:07.722103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-433a3e1c-b47c-4332-9763-ff578f9f3c97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:07.722807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-433a3e1c-b47c-4332-9763-ff578f9f3c97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:07.724986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1126/2260] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:22:07 2022] DELETE /identity/v3/projects/c61d4ea2b52c497caeb298fe8677488e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:07.730509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa650d9-7528-4c7c-8ca7-ddcaa688dd25 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:07.730875 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa650d9-7528-4c7c-8ca7-ddcaa688dd25 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:07.731223 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa650d9-7528-4c7c-8ca7-ddcaa688dd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:07.751692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-daa650d9-7528-4c7c-8ca7-ddcaa688dd25 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:07.752959 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-daa650d9-7528-4c7c-8ca7-ddcaa688dd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:07.815665 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1135/2261] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:07.823825 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c30ab-69f4-46e8-84a5-8fea6bede903 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:07.824117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c30ab-69f4-46e8-84a5-8fea6bede903 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:07.824415 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c30ab-69f4-46e8-84a5-8fea6bede903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:07.845248 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-bb0c30ab-69f4-46e8-84a5-8fea6bede903 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:07.846495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-bb0c30ab-69f4-46e8-84a5-8fea6bede903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:07.854690 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1127/2262] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:07.859089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa650d9-7528-4c7c-8ca7-ddcaa688dd25 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:07.870352 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa650d9-7528-4c7c-8ca7-ddcaa688dd25 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:07.875755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6688a-6794-4ac3-ab7b-9d7378d4a540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:07.876386 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6688a-6794-4ac3-ab7b-9d7378d4a540 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:07.876690 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6688a-6794-4ac3-ab7b-9d7378d4a540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:07.876969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6688a-6794-4ac3-ab7b-9d7378d4a540 admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:07.877679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6688a-6794-4ac3-ab7b-9d7378d4a540 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:22:07.877986 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6688a-6794-4ac3-ab7b-9d7378d4a540 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:07.880946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d6688a-6794-4ac3-ab7b-9d7378d4a540 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:07.895606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1136/2263] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:22:07.899991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c30ab-69f4-46e8-84a5-8fea6bede903 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:07.909851 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c30ab-69f4-46e8-84a5-8fea6bede903 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:07.915244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c805037-e3e2-416f-97e1-60c4e5a1e1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:07.915871 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c805037-e3e2-416f-97e1-60c4e5a1e1d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:07.916179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c805037-e3e2-416f-97e1-60c4e5a1e1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:07.916456 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c805037-e3e2-416f-97e1-60c4e5a1e1d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:07.917188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c805037-e3e2-416f-97e1-60c4e5a1e1d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:07.919121 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c805037-e3e2-416f-97e1-60c4e5a1e1d8 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:07.949690 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1128/2264] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:22:07 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:22:07.953911 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6688a-6794-4ac3-ab7b-9d7378d4a540 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:07.964330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6688a-6794-4ac3-ab7b-9d7378d4a540 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:08.004751 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:08.045193 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc068842-f021-4a17-9a9f-83e25d0b8591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:08.045939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc068842-f021-4a17-9a9f-83e25d0b8591 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:08.046363 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc068842-f021-4a17-9a9f-83e25d0b8591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:08.046776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc068842-f021-4a17-9a9f-83e25d0b8591 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:08.047532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc068842-f021-4a17-9a9f-83e25d0b8591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:08.049509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc068842-f021-4a17-9a9f-83e25d0b8591 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:08.073838 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1137/2265] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:22:07 2022] POST /identity/v3/users => generated 285 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:22:08.081575 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c805037-e3e2-416f-97e1-60c4e5a1e1d8 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:08.092756 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c805037-e3e2-416f-97e1-60c4e5a1e1d8 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:08.097606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be75ad-d158-4ae9-8d84-a5ad77f54e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:08.098282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be75ad-d158-4ae9-8d84-a5ad77f54e95 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:08.098564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be75ad-d158-4ae9-8d84-a5ad77f54e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:08.099336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be75ad-d158-4ae9-8d84-a5ad77f54e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:08.100079 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be75ad-d158-4ae9-8d84-a5ad77f54e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:08.101771 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be75ad-d158-4ae9-8d84-a5ad77f54e95 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:08.111994 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1129/2266] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:22:08.116855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc068842-f021-4a17-9a9f-83e25d0b8591 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:08.126764 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc068842-f021-4a17-9a9f-83e25d0b8591 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:08.131649 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d99a5e-f2c4-452e-a308-25ce41d00163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:08.132544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d99a5e-f2c4-452e-a308-25ce41d00163 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:08.133070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d99a5e-f2c4-452e-a308-25ce41d00163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:08.133556 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d99a5e-f2c4-452e-a308-25ce41d00163 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:08.134841 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d99a5e-f2c4-452e-a308-25ce41d00163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:08.137450 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d99a5e-f2c4-452e-a308-25ce41d00163 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:08.150014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1138/2267] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:22:08.154909 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be75ad-d158-4ae9-8d84-a5ad77f54e95 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:08.167460 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be75ad-d158-4ae9-8d84-a5ad77f54e95 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:08.172459 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fab58-92e6-4c74-befc-c119c891fad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:08.173150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fab58-92e6-4c74-befc-c119c891fad6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:08.173435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fab58-92e6-4c74-befc-c119c891fad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:08.173711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fab58-92e6-4c74-befc-c119c891fad6 admin admin] PATH_INFO: `/v3/projects/7d1b25acf4d4415ba28194c73b2d7ba9/users/65a0ab4c141b4f9481107cae6d8b192e/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:08.189012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fab58-92e6-4c74-befc-c119c891fad6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d1b25acf4d4415ba28194c73b2d7ba9, user_id=65a0ab4c141b4f9481107cae6d8b192e, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:08.191679 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fab58-92e6-4c74-befc-c119c891fad6 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:08.204969 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1130/2268] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:22:08 2022] PUT /identity/v3/projects/7d1b25acf4d4415ba28194c73b2d7ba9/users/65a0ab4c141b4f9481107cae6d8b192e/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:08.215459 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba56e9e-33f0-4b95-8248-b8b0e08bf2e6 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:08.215775 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba56e9e-33f0-4b95-8248-b8b0e08bf2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:08.216052 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba56e9e-33f0-4b95-8248-b8b0e08bf2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:08.244396 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-bba56e9e-33f0-4b95-8248-b8b0e08bf2e6 None None] MFA Rules not processed for user `65a0ab4c141b4f9481107cae6d8b192e`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:08.245665 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-bba56e9e-33f0-4b95-8248-b8b0e08bf2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:08.311438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1139/2269] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:08.315902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fab58-92e6-4c74-befc-c119c891fad6 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:08.372300 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:08.379170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fab58-92e6-4c74-befc-c119c891fad6 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:08.384823 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b46d4-19ef-49a0-a666-b8a52bf359f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:08.385468 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b46d4-19ef-49a0-a666-b8a52bf359f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:08.385752 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b46d4-19ef-49a0-a666-b8a52bf359f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:08.386054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b46d4-19ef-49a0-a666-b8a52bf359f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:08.386728 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b46d4-19ef-49a0-a666-b8a52bf359f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:08.388652 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b46d4-19ef-49a0-a666-b8a52bf359f9 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:08.420556 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1131/2270] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:22:08 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:22:08.421780 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba56e9e-33f0-4b95-8248-b8b0e08bf2e6 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:08.432952 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba56e9e-33f0-4b95-8248-b8b0e08bf2e6 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:08.506585 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0681a1c-2132-4d79-ac5e-30248f06b025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:08.507219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0681a1c-2132-4d79-ac5e-30248f06b025 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:08.507507 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0681a1c-2132-4d79-ac5e-30248f06b025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:08.507785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0681a1c-2132-4d79-ac5e-30248f06b025 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:08.508455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0681a1c-2132-4d79-ac5e-30248f06b025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:08.510545 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0681a1c-2132-4d79-ac5e-30248f06b025 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:08.537828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1140/2271] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:22:08 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:22:08.539728 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b46d4-19ef-49a0-a666-b8a52bf359f9 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:08.550281 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b46d4-19ef-49a0-a666-b8a52bf359f9 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:08.555277 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de3d02-bf48-4916-9280-56d6410864d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:08.555989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de3d02-bf48-4916-9280-56d6410864d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:08.556337 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de3d02-bf48-4916-9280-56d6410864d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:08.556711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de3d02-bf48-4916-9280-56d6410864d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:08.557540 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de3d02-bf48-4916-9280-56d6410864d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:08.559105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de3d02-bf48-4916-9280-56d6410864d7 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:08.570966 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1132/2272] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:22:08.575131 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0681a1c-2132-4d79-ac5e-30248f06b025 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:08.585754 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0681a1c-2132-4d79-ac5e-30248f06b025 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:08.591003 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07623c5f-1bdc-426e-b264-d81de17ecdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:08.591648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07623c5f-1bdc-426e-b264-d81de17ecdcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:08.591939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07623c5f-1bdc-426e-b264-d81de17ecdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:08.592209 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07623c5f-1bdc-426e-b264-d81de17ecdcb admin admin] PATH_INFO: `/v3/projects/4547a3c13911403389b0229be9d1a7b2/users/55e7581278bd4cd9a2409b61e2ea6555/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:08.607330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07623c5f-1bdc-426e-b264-d81de17ecdcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4547a3c13911403389b0229be9d1a7b2, user_id=55e7581278bd4cd9a2409b61e2ea6555, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:08.610029 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07623c5f-1bdc-426e-b264-d81de17ecdcb admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:08.623130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1141/2273] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:22:08 2022] PUT /identity/v3/projects/4547a3c13911403389b0229be9d1a7b2/users/55e7581278bd4cd9a2409b61e2ea6555/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:08.627360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de3d02-bf48-4916-9280-56d6410864d7 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:08.637516 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de3d02-bf48-4916-9280-56d6410864d7 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:08.696551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104d489-a1d5-4d4a-a7c4-3680be337d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:08.697197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104d489-a1d5-4d4a-a7c4-3680be337d6f admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:08.697559 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104d489-a1d5-4d4a-a7c4-3680be337d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:08.697752 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104d489-a1d5-4d4a-a7c4-3680be337d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:08.698443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104d489-a1d5-4d4a-a7c4-3680be337d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:08.700048 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104d489-a1d5-4d4a-a7c4-3680be337d6f admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:08.710515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1133/2274] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:22:08.714800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07623c5f-1bdc-426e-b264-d81de17ecdcb admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:08.725553 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07623c5f-1bdc-426e-b264-d81de17ecdcb admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:08.730906 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c477e-3635-45d5-87c4-2dbdc5d91f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:08.731613 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c477e-3635-45d5-87c4-2dbdc5d91f05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:08.731898 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c477e-3635-45d5-87c4-2dbdc5d91f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:08.732177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c477e-3635-45d5-87c4-2dbdc5d91f05 admin admin] PATH_INFO: `/v3/projects/4547a3c13911403389b0229be9d1a7b2/users/55e7581278bd4cd9a2409b61e2ea6555/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:08.734217 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c477e-3635-45d5-87c4-2dbdc5d91f05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4547a3c13911403389b0229be9d1a7b2, user_id=55e7581278bd4cd9a2409b61e2ea6555, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:08.736942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c477e-3635-45d5-87c4-2dbdc5d91f05 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:08.739294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:08.749424 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1142/2275] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:22:08 2022] PUT /identity/v3/projects/4547a3c13911403389b0229be9d1a7b2/users/55e7581278bd4cd9a2409b61e2ea6555/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:08.761421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5522b4e-92cc-466a-aead-7134e4f07697 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:08.761693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5522b4e-92cc-466a-aead-7134e4f07697 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:08.761968 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5522b4e-92cc-466a-aead-7134e4f07697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:08.790865 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-f5522b4e-92cc-466a-aead-7134e4f07697 None None] MFA Rules not processed for user `55e7581278bd4cd9a2409b61e2ea6555`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:08.792084 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-f5522b4e-92cc-466a-aead-7134e4f07697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:08.857827 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1134/2276] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:08 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:08.868812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c477e-3635-45d5-87c4-2dbdc5d91f05 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:08.870415 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-fc2c477e-3635-45d5-87c4-2dbdc5d91f05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:08.927730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c477e-3635-45d5-87c4-2dbdc5d91f05 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:08.931428 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59265a-5ef0-47bb-b7af-79708223e6aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:08.932045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59265a-5ef0-47bb-b7af-79708223e6aa service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:08.932328 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59265a-5ef0-47bb-b7af-79708223e6aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:08.932612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59265a-5ef0-47bb-b7af-79708223e6aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:08.941519 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59265a-5ef0-47bb-b7af-79708223e6aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:08.943475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59265a-5ef0-47bb-b7af-79708223e6aa service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:08.954409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1143/2277] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:22:08 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:22:10.360813 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5522b4e-92cc-466a-aead-7134e4f07697 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:10.415301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:10.428411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5522b4e-92cc-466a-aead-7134e4f07697 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:10.433177 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ac4f9-59f0-44e7-a5a0-bec27f4a7ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:10.434041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ac4f9-59f0-44e7-a5a0-bec27f4a7ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:10.434323 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ac4f9-59f0-44e7-a5a0-bec27f4a7ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:10.434597 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ac4f9-59f0-44e7-a5a0-bec27f4a7ec3 admin admin] PATH_INFO: `/v3/users/65a0ab4c141b4f9481107cae6d8b192e` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:10.435812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ac4f9-59f0-44e7-a5a0-bec27f4a7ec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65a0ab4c141b4f9481107cae6d8b192e)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:10.437776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ac4f9-59f0-44e7-a5a0-bec27f4a7ec3 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:10.484839 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b6ac4f9-59f0-44e7-a5a0-bec27f4a7ec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65a0ab4c141b4f9481107cae6d8b192e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:10.485049 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b6ac4f9-59f0-44e7-a5a0-bec27f4a7ec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65a0ab4c141b4f9481107cae6d8b192e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:10.485648 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b6ac4f9-59f0-44e7-a5a0-bec27f4a7ec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65a0ab4c141b4f9481107cae6d8b192e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:10.497500 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b6ac4f9-59f0-44e7-a5a0-bec27f4a7ec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65a0ab4c141b4f9481107cae6d8b192e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:10.504207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b6ac4f9-59f0-44e7-a5a0-bec27f4a7ec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65a0ab4c141b4f9481107cae6d8b192e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:10.518246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1135/2278] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:22:10 2022] DELETE /identity/v3/users/65a0ab4c141b4f9481107cae6d8b192e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:10.523739 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49df8b1-7f51-4e43-b53d-7ee01574e323 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:10.524215 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49df8b1-7f51-4e43-b53d-7ee01574e323 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:10.524376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49df8b1-7f51-4e43-b53d-7ee01574e323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:10.543113 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-a49df8b1-7f51-4e43-b53d-7ee01574e323 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:10.544310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a49df8b1-7f51-4e43-b53d-7ee01574e323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:10.602344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1144/2279] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:10.612219 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ac4f9-59f0-44e7-a5a0-bec27f4a7ec3 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:10.613658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-8b6ac4f9-59f0-44e7-a5a0-bec27f4a7ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:10.671811 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ac4f9-59f0-44e7-a5a0-bec27f4a7ec3 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:10.675466 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f7995-a3d7-4c0b-82ba-25342fcfa1f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:10.676086 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f7995-a3d7-4c0b-82ba-25342fcfa1f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:10.676369 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f7995-a3d7-4c0b-82ba-25342fcfa1f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:10.676666 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f7995-a3d7-4c0b-82ba-25342fcfa1f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:10.686228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f7995-a3d7-4c0b-82ba-25342fcfa1f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:10.687674 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f7995-a3d7-4c0b-82ba-25342fcfa1f5 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:10.698268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1136/2280] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:22:10.987226 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49df8b1-7f51-4e43-b53d-7ee01574e323 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:10.988766 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a49df8b1-7f51-4e43-b53d-7ee01574e323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:11.001784 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49df8b1-7f51-4e43-b53d-7ee01574e323 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:11.007154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b187ec-03d2-4a20-ad55-8b255081828d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:11.007809 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b187ec-03d2-4a20-ad55-8b255081828d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:11.008087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b187ec-03d2-4a20-ad55-8b255081828d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:11.008360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b187ec-03d2-4a20-ad55-8b255081828d admin admin] PATH_INFO: `/v3/projects/7d1b25acf4d4415ba28194c73b2d7ba9` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:11.009549 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b187ec-03d2-4a20-ad55-8b255081828d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d1b25acf4d4415ba28194c73b2d7ba9)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:11.011449 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b187ec-03d2-4a20-ad55-8b255081828d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:11.058852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-60b187ec-03d2-4a20-ad55-8b255081828d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d1b25acf4d4415ba28194c73b2d7ba9', 'request_id': 'req-60b187ec-03d2-4a20-ad55-8b255081828d'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:11.065984 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-60b187ec-03d2-4a20-ad55-8b255081828d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d1b25acf4d4415ba28194c73b2d7ba9', 'request_id': 'req-60b187ec-03d2-4a20-ad55-8b255081828d'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:11.071667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-60b187ec-03d2-4a20-ad55-8b255081828d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d1b25acf4d4415ba28194c73b2d7ba9', 'request_id': 'req-60b187ec-03d2-4a20-ad55-8b255081828d'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:11.082355 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-60b187ec-03d2-4a20-ad55-8b255081828d admin admin] The token cache is being invalidate because project 7d1b25acf4d4415ba28194c73b2d7ba9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:22:11.082635 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-60b187ec-03d2-4a20-ad55-8b255081828d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:11.083270 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-60b187ec-03d2-4a20-ad55-8b255081828d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:11.084796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1145/2281] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:22:10 2022] DELETE /identity/v3/projects/7d1b25acf4d4415ba28194c73b2d7ba9 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:11.088838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f7995-a3d7-4c0b-82ba-25342fcfa1f5 service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:11.099356 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f7995-a3d7-4c0b-82ba-25342fcfa1f5 service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:11.155293 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1e3f0-714c-48be-abef-e254e3dbe92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:11.155933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1e3f0-714c-48be-abef-e254e3dbe92f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:11.156213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1e3f0-714c-48be-abef-e254e3dbe92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:11.156488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1e3f0-714c-48be-abef-e254e3dbe92f admin admin] PATH_INFO: `/v3/users/55e7581278bd4cd9a2409b61e2ea6555` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:11.157679 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1e3f0-714c-48be-abef-e254e3dbe92f admin admin] RBAC: Authorizing `identity:delete_user(user_id=55e7581278bd4cd9a2409b61e2ea6555)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:11.159613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1e3f0-714c-48be-abef-e254e3dbe92f admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:11.173745 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:11.210007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-6ce1e3f0-714c-48be-abef-e254e3dbe92f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55e7581278bd4cd9a2409b61e2ea6555'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:11.210662 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-6ce1e3f0-714c-48be-abef-e254e3dbe92f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55e7581278bd4cd9a2409b61e2ea6555'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:11.211272 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-6ce1e3f0-714c-48be-abef-e254e3dbe92f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55e7581278bd4cd9a2409b61e2ea6555'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:11.223547 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-6ce1e3f0-714c-48be-abef-e254e3dbe92f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55e7581278bd4cd9a2409b61e2ea6555'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:11.232325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-6ce1e3f0-714c-48be-abef-e254e3dbe92f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55e7581278bd4cd9a2409b61e2ea6555'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:11.247024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1137/2282] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:22:11 2022] DELETE /identity/v3/users/55e7581278bd4cd9a2409b61e2ea6555 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:11.575133 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b187ec-03d2-4a20-ad55-8b255081828d admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:11.579323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-60b187ec-03d2-4a20-ad55-8b255081828d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:11.619759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b187ec-03d2-4a20-ad55-8b255081828d admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:11.657266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:11.719784 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4887bf8-63f8-4f97-96e2-a994d8219da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:11.720613 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4887bf8-63f8-4f97-96e2-a994d8219da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:11.721021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4887bf8-63f8-4f97-96e2-a994d8219da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:11.721391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4887bf8-63f8-4f97-96e2-a994d8219da9 admin admin] PATH_INFO: `/v3/projects/4547a3c13911403389b0229be9d1a7b2` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:11.724980 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4887bf8-63f8-4f97-96e2-a994d8219da9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4547a3c13911403389b0229be9d1a7b2)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:11.724980 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4887bf8-63f8-4f97-96e2-a994d8219da9 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:11.787654 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b4887bf8-63f8-4f97-96e2-a994d8219da9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4547a3c13911403389b0229be9d1a7b2', 'request_id': 'req-b4887bf8-63f8-4f97-96e2-a994d8219da9'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:11.798321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b4887bf8-63f8-4f97-96e2-a994d8219da9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4547a3c13911403389b0229be9d1a7b2', 'request_id': 'req-b4887bf8-63f8-4f97-96e2-a994d8219da9'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:11.804302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b4887bf8-63f8-4f97-96e2-a994d8219da9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4547a3c13911403389b0229be9d1a7b2', 'request_id': 'req-b4887bf8-63f8-4f97-96e2-a994d8219da9'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:11.815479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b4887bf8-63f8-4f97-96e2-a994d8219da9 admin admin] The token cache is being invalidate because project 4547a3c13911403389b0229be9d1a7b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:22:11.815855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b4887bf8-63f8-4f97-96e2-a994d8219da9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:11.816597 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b4887bf8-63f8-4f97-96e2-a994d8219da9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:11.818975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1146/2283] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:22:11 2022] DELETE /identity/v3/projects/4547a3c13911403389b0229be9d1a7b2 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:11.823931 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab2660-9daf-4ca3-a8be-3d3b2632afb2 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:11.824390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab2660-9daf-4ca3-a8be-3d3b2632afb2 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:11.824512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab2660-9daf-4ca3-a8be-3d3b2632afb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:11.844649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-fbab2660-9daf-4ca3-a8be-3d3b2632afb2 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:11.845895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-fbab2660-9daf-4ca3-a8be-3d3b2632afb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:11.903630 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1138/2284] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:11.911645 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ffc71-d511-4879-8b0d-6b4e03afe35f None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:11.911925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ffc71-d511-4879-8b0d-6b4e03afe35f None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:11.912198 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784ffc71-d511-4879-8b0d-6b4e03afe35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:11.932729 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-784ffc71-d511-4879-8b0d-6b4e03afe35f None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:11.933938 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-784ffc71-d511-4879-8b0d-6b4e03afe35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:11.941693 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1147/2285] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:11.946522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab2660-9daf-4ca3-a8be-3d3b2632afb2 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:11.956228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab2660-9daf-4ca3-a8be-3d3b2632afb2 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:11.961465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b94c94-a9ce-4bbd-a5b2-09d2c29793a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:11.962075 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b94c94-a9ce-4bbd-a5b2-09d2c29793a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:11.962361 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b94c94-a9ce-4bbd-a5b2-09d2c29793a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:11.962635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b94c94-a9ce-4bbd-a5b2-09d2c29793a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:11.963333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b94c94-a9ce-4bbd-a5b2-09d2c29793a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:22:11.963641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b94c94-a9ce-4bbd-a5b2-09d2c29793a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:11.965281 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b94c94-a9ce-4bbd-a5b2-09d2c29793a5 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:11.977516 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1139/2286] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:22:11.982478 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ffc71-d511-4879-8b0d-6b4e03afe35f None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:11.991923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784ffc71-d511-4879-8b0d-6b4e03afe35f None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:11.996595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17cfd1-d802-44fc-a1bf-46f0b334a8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:11.997238 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17cfd1-d802-44fc-a1bf-46f0b334a8d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:11.997523 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17cfd1-d802-44fc-a1bf-46f0b334a8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:11.997799 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f17cfd1-d802-44fc-a1bf-46f0b334a8d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:11.998488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f17cfd1-d802-44fc-a1bf-46f0b334a8d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:12.000397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f17cfd1-d802-44fc-a1bf-46f0b334a8d3 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:12.029786 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1148/2287] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:22:11 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:22:12.033830 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b94c94-a9ce-4bbd-a5b2-09d2c29793a5 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:12.046553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b94c94-a9ce-4bbd-a5b2-09d2c29793a5 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:12.108986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7233c70-7151-4765-899f-61d4fcef1c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:12.109607 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7233c70-7151-4765-899f-61d4fcef1c33 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:12.109888 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7233c70-7151-4765-899f-61d4fcef1c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:12.110177 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7233c70-7151-4765-899f-61d4fcef1c33 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:12.110849 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7233c70-7151-4765-899f-61d4fcef1c33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:12.112810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7233c70-7151-4765-899f-61d4fcef1c33 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:12.139898 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1140/2288] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:22:12 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:22:12.141138 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17cfd1-d802-44fc-a1bf-46f0b334a8d3 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:12.151872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f17cfd1-d802-44fc-a1bf-46f0b334a8d3 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:12.156693 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e17792-d0ab-42d0-b9ba-cdfd7feb1dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:12.157250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e17792-d0ab-42d0-b9ba-cdfd7feb1dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:12.157531 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e17792-d0ab-42d0-b9ba-cdfd7feb1dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:12.157804 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e17792-d0ab-42d0-b9ba-cdfd7feb1dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:12.158493 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e17792-d0ab-42d0-b9ba-cdfd7feb1dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:12.160068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e17792-d0ab-42d0-b9ba-cdfd7feb1dc1 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:12.171447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1149/2289] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:22:12.175539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7233c70-7151-4765-899f-61d4fcef1c33 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:12.185634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7233c70-7151-4765-899f-61d4fcef1c33 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:12.190558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ffbfe-795d-4f81-9ccd-42c53547c08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:12.191184 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ffbfe-795d-4f81-9ccd-42c53547c08c admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:12.191466 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ffbfe-795d-4f81-9ccd-42c53547c08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:12.191738 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ffbfe-795d-4f81-9ccd-42c53547c08c admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:12.192486 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ffbfe-795d-4f81-9ccd-42c53547c08c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:12.194124 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ffbfe-795d-4f81-9ccd-42c53547c08c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:12.196401 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:12.207711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1141/2290] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:22:12.212395 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e17792-d0ab-42d0-b9ba-cdfd7feb1dc1 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:12.222548 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e17792-d0ab-42d0-b9ba-cdfd7feb1dc1 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:12.227774 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e446f-6fa3-4b1a-821e-d4e9609af627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:12.228422 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e446f-6fa3-4b1a-821e-d4e9609af627 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:12.228727 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e446f-6fa3-4b1a-821e-d4e9609af627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:12.229023 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e446f-6fa3-4b1a-821e-d4e9609af627 admin admin] PATH_INFO: `/v3/projects/dc4b7b6e1399482b92bbb6b007e1eb35/users/f1706796970444cdbd09f129336de790/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:12.243145 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e446f-6fa3-4b1a-821e-d4e9609af627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc4b7b6e1399482b92bbb6b007e1eb35, user_id=f1706796970444cdbd09f129336de790, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:12.245817 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e446f-6fa3-4b1a-821e-d4e9609af627 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:12.258572 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1150/2291] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:22:12 2022] PUT /identity/v3/projects/dc4b7b6e1399482b92bbb6b007e1eb35/users/f1706796970444cdbd09f129336de790/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:12.270145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90473495-98be-4c72-af23-88db8f7d8e76 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:12.270429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90473495-98be-4c72-af23-88db8f7d8e76 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:12.271193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90473495-98be-4c72-af23-88db8f7d8e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:12.300125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-90473495-98be-4c72-af23-88db8f7d8e76 None None] MFA Rules not processed for user `f1706796970444cdbd09f129336de790`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:12.301376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-90473495-98be-4c72-af23-88db8f7d8e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:12.366137 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1142/2292] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:12.370629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e446f-6fa3-4b1a-821e-d4e9609af627 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:12.419924 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:12.436284 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e446f-6fa3-4b1a-821e-d4e9609af627 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:12.441064 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853b331-e2e6-49e6-94a5-efe34fa7fd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:12.441674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853b331-e2e6-49e6-94a5-efe34fa7fd77 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:12.441962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853b331-e2e6-49e6-94a5-efe34fa7fd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:12.442266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853b331-e2e6-49e6-94a5-efe34fa7fd77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:12.443432 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853b331-e2e6-49e6-94a5-efe34fa7fd77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:12.445388 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d853b331-e2e6-49e6-94a5-efe34fa7fd77 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:12.471897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1151/2293] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:22:12 2022] POST /identity/v3/projects => generated 381 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:22:12.476033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90473495-98be-4c72-af23-88db8f7d8e76 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:12.487460 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90473495-98be-4c72-af23-88db8f7d8e76 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:12.543128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1838d-4583-4267-bee3-5ef6d38bd563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:12.543760 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1838d-4583-4267-bee3-5ef6d38bd563 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:12.544023 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1838d-4583-4267-bee3-5ef6d38bd563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:12.544293 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1838d-4583-4267-bee3-5ef6d38bd563 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:12.544981 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1838d-4583-4267-bee3-5ef6d38bd563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:12.546884 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1838d-4583-4267-bee3-5ef6d38bd563 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:12.568005 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1143/2294] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:22:12 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:22:12.572297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853b331-e2e6-49e6-94a5-efe34fa7fd77 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:12.582509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853b331-e2e6-49e6-94a5-efe34fa7fd77 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:12.587666 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff61d8-eb74-4620-90f6-5e931abc6d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:12.588266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff61d8-eb74-4620-90f6-5e931abc6d6c admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:12.588547 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff61d8-eb74-4620-90f6-5e931abc6d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:12.588851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff61d8-eb74-4620-90f6-5e931abc6d6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:12.589568 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff61d8-eb74-4620-90f6-5e931abc6d6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:12.591408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ff61d8-eb74-4620-90f6-5e931abc6d6c admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:12.602322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1152/2295] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:22:12.606521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1838d-4583-4267-bee3-5ef6d38bd563 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:12.616433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1838d-4583-4267-bee3-5ef6d38bd563 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:12.621243 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f79498-bc26-4b36-b085-ef2b14213c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:12.621928 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f79498-bc26-4b36-b085-ef2b14213c92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:12.622215 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f79498-bc26-4b36-b085-ef2b14213c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:12.622534 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f79498-bc26-4b36-b085-ef2b14213c92 admin admin] PATH_INFO: `/v3/projects/36e111682adc40948af455a92fadaa0a/users/ff2ab192f1ba4ba4936dbd0b6aa4306f/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:12.637879 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f79498-bc26-4b36-b085-ef2b14213c92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36e111682adc40948af455a92fadaa0a, user_id=ff2ab192f1ba4ba4936dbd0b6aa4306f, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:12.640588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f79498-bc26-4b36-b085-ef2b14213c92 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:12.654969 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1144/2296] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:22:12 2022] PUT /identity/v3/projects/36e111682adc40948af455a92fadaa0a/users/ff2ab192f1ba4ba4936dbd0b6aa4306f/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:22:12.659053 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff61d8-eb74-4620-90f6-5e931abc6d6c admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:12.669274 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff61d8-eb74-4620-90f6-5e931abc6d6c admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:12.724513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961d4a4-5889-45a9-be46-5ee485e235fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:12.725169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961d4a4-5889-45a9-be46-5ee485e235fb admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:12.725452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961d4a4-5889-45a9-be46-5ee485e235fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:12.725730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961d4a4-5889-45a9-be46-5ee485e235fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:12.726430 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f961d4a4-5889-45a9-be46-5ee485e235fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:12.728026 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f961d4a4-5889-45a9-be46-5ee485e235fb admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:12.743042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1153/2297] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:22:12.744241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f79498-bc26-4b36-b085-ef2b14213c92 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:12.756303 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f79498-bc26-4b36-b085-ef2b14213c92 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:12.761075 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8a4f6-90a6-4bef-83cb-afe71f31b818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:12.761819 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8a4f6-90a6-4bef-83cb-afe71f31b818 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:12.762162 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8a4f6-90a6-4bef-83cb-afe71f31b818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:12.762498 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8a4f6-90a6-4bef-83cb-afe71f31b818 admin admin] PATH_INFO: `/v3/projects/36e111682adc40948af455a92fadaa0a/users/ff2ab192f1ba4ba4936dbd0b6aa4306f/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:12.764552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f8a4f6-90a6-4bef-83cb-afe71f31b818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36e111682adc40948af455a92fadaa0a, user_id=ff2ab192f1ba4ba4936dbd0b6aa4306f, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:12.767359 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f8a4f6-90a6-4bef-83cb-afe71f31b818 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:12.779364 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1145/2298] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:22:12 2022] PUT /identity/v3/projects/36e111682adc40948af455a92fadaa0a/users/ff2ab192f1ba4ba4936dbd0b6aa4306f/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:12.789315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2764d5-1897-4aa2-abc0-b09f84d900d4 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:12.789602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2764d5-1897-4aa2-abc0-b09f84d900d4 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:12.789882 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2764d5-1897-4aa2-abc0-b09f84d900d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:12.819577 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-8d2764d5-1897-4aa2-abc0-b09f84d900d4 None None] MFA Rules not processed for user `ff2ab192f1ba4ba4936dbd0b6aa4306f`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:12.820815 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8d2764d5-1897-4aa2-abc0-b09f84d900d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:12.890703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1154/2299] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:12.902082 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8a4f6-90a6-4bef-83cb-afe71f31b818 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:12.904083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-16f8a4f6-90a6-4bef-83cb-afe71f31b818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:12.947358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:12.963515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8a4f6-90a6-4bef-83cb-afe71f31b818 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:12.967480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4844f-e706-4d79-a765-6316bcd3726b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:12.968097 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4844f-e706-4d79-a765-6316bcd3726b service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:12.968378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4844f-e706-4d79-a765-6316bcd3726b service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:12.968678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4844f-e706-4d79-a765-6316bcd3726b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:12.978939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4844f-e706-4d79-a765-6316bcd3726b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:12.980297 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4844f-e706-4d79-a765-6316bcd3726b service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:12.989980 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1146/2300] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:22:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:22:13.411609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2764d5-1897-4aa2-abc0-b09f84d900d4 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:13.457007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:13.477033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2764d5-1897-4aa2-abc0-b09f84d900d4 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:13.481608 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf3388-0db9-49b0-ad70-b7178e1c1685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:13.482252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf3388-0db9-49b0-ad70-b7178e1c1685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:13.482532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf3388-0db9-49b0-ad70-b7178e1c1685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:13.482808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf3388-0db9-49b0-ad70-b7178e1c1685 admin admin] PATH_INFO: `/v3/users/f1706796970444cdbd09f129336de790` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:13.483979 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf3388-0db9-49b0-ad70-b7178e1c1685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1706796970444cdbd09f129336de790)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:13.485923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecf3388-0db9-49b0-ad70-b7178e1c1685 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:13.547927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3ecf3388-0db9-49b0-ad70-b7178e1c1685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1706796970444cdbd09f129336de790'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:13.556326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3ecf3388-0db9-49b0-ad70-b7178e1c1685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1706796970444cdbd09f129336de790'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:13.557092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3ecf3388-0db9-49b0-ad70-b7178e1c1685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1706796970444cdbd09f129336de790'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:13.569621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3ecf3388-0db9-49b0-ad70-b7178e1c1685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1706796970444cdbd09f129336de790'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:13.570322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3ecf3388-0db9-49b0-ad70-b7178e1c1685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1706796970444cdbd09f129336de790'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:13.583075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1155/2301] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:22:13 2022] DELETE /identity/v3/users/f1706796970444cdbd09f129336de790 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:13.587613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037006ae-476e-4665-a436-3396dbb36353 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:13.587897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037006ae-476e-4665-a436-3396dbb36353 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:13.588166 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037006ae-476e-4665-a436-3396dbb36353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:13.607155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-037006ae-476e-4665-a436-3396dbb36353 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:13.608343 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-037006ae-476e-4665-a436-3396dbb36353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:13.667479 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1147/2302] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:13.677524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf3388-0db9-49b0-ad70-b7178e1c1685 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:13.678920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-3ecf3388-0db9-49b0-ad70-b7178e1c1685 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:13.734785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecf3388-0db9-49b0-ad70-b7178e1c1685 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:13.738373 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd29413-c878-42ad-b84a-eadde287af5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:13.738978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd29413-c878-42ad-b84a-eadde287af5e service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:13.739786 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd29413-c878-42ad-b84a-eadde287af5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:13.740058 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd29413-c878-42ad-b84a-eadde287af5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:13.749028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd29413-c878-42ad-b84a-eadde287af5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:13.750443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd29413-c878-42ad-b84a-eadde287af5e service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:13.760337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1156/2303] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:22:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:22:13.947730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037006ae-476e-4665-a436-3396dbb36353 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:13.949321 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-037006ae-476e-4665-a436-3396dbb36353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:13.961888 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037006ae-476e-4665-a436-3396dbb36353 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:13.965872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0ef74-74b1-44d2-aa52-14505e741c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:13.966489 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0ef74-74b1-44d2-aa52-14505e741c84 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:13.966768 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0ef74-74b1-44d2-aa52-14505e741c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:13.967110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0ef74-74b1-44d2-aa52-14505e741c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:13.969145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-18e0ef74-74b1-44d2-aa52-14505e741c84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:14.031235 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0ef74-74b1-44d2-aa52-14505e741c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:14.033326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0ef74-74b1-44d2-aa52-14505e741c84 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:14.045533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1148/2304] 10.222.0.45 () {60 vars in 1396 bytes} [Thu Aug 4 02:22:13 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:22:14.046687 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd29413-c878-42ad-b84a-eadde287af5e service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:14.048185 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-fbd29413-c878-42ad-b84a-eadde287af5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:14.061800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd29413-c878-42ad-b84a-eadde287af5e service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:14.066335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa1113-d940-448c-b399-f22c6b9bb12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:14.066962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa1113-d940-448c-b399-f22c6b9bb12f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:14.067241 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa1113-d940-448c-b399-f22c6b9bb12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:14.068133 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa1113-d940-448c-b399-f22c6b9bb12f admin admin] PATH_INFO: `/v3/projects/dc4b7b6e1399482b92bbb6b007e1eb35` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:14.069320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa1113-d940-448c-b399-f22c6b9bb12f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc4b7b6e1399482b92bbb6b007e1eb35)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:14.071207 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa1113-d940-448c-b399-f22c6b9bb12f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:14.133482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-c1fa1113-d940-448c-b399-f22c6b9bb12f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc4b7b6e1399482b92bbb6b007e1eb35', 'request_id': 'req-c1fa1113-d940-448c-b399-f22c6b9bb12f'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:14.141667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-c1fa1113-d940-448c-b399-f22c6b9bb12f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc4b7b6e1399482b92bbb6b007e1eb35', 'request_id': 'req-c1fa1113-d940-448c-b399-f22c6b9bb12f'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:14.144216 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:14.149756 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-c1fa1113-d940-448c-b399-f22c6b9bb12f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc4b7b6e1399482b92bbb6b007e1eb35', 'request_id': 'req-c1fa1113-d940-448c-b399-f22c6b9bb12f'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:14.162940 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-c1fa1113-d940-448c-b399-f22c6b9bb12f admin admin] The token cache is being invalidate because project dc4b7b6e1399482b92bbb6b007e1eb35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:22:14.163230 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-c1fa1113-d940-448c-b399-f22c6b9bb12f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:14.163894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-c1fa1113-d940-448c-b399-f22c6b9bb12f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:14.165543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1157/2305] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:22:14 2022] DELETE /identity/v3/projects/dc4b7b6e1399482b92bbb6b007e1eb35 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:22:14.169585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0ef74-74b1-44d2-aa52-14505e741c84 service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:14.181306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0ef74-74b1-44d2-aa52-14505e741c84 service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:14.241387 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1012b5-79d1-443f-906e-770e5da415e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:14.242037 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1012b5-79d1-443f-906e-770e5da415e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:14.242316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1012b5-79d1-443f-906e-770e5da415e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:14.242587 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1012b5-79d1-443f-906e-770e5da415e7 admin admin] PATH_INFO: `/v3/users/ff2ab192f1ba4ba4936dbd0b6aa4306f` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:14.243777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1012b5-79d1-443f-906e-770e5da415e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff2ab192f1ba4ba4936dbd0b6aa4306f)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:14.245751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1012b5-79d1-443f-906e-770e5da415e7 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:14.269157 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:14.297619 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b1012b5-79d1-443f-906e-770e5da415e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff2ab192f1ba4ba4936dbd0b6aa4306f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:14.298398 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b1012b5-79d1-443f-906e-770e5da415e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff2ab192f1ba4ba4936dbd0b6aa4306f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:14.299385 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b1012b5-79d1-443f-906e-770e5da415e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff2ab192f1ba4ba4936dbd0b6aa4306f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:14.318471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b1012b5-79d1-443f-906e-770e5da415e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff2ab192f1ba4ba4936dbd0b6aa4306f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:14.327015 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8b1012b5-79d1-443f-906e-770e5da415e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff2ab192f1ba4ba4936dbd0b6aa4306f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:14.341970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1149/2306] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:22:14 2022] DELETE /identity/v3/users/ff2ab192f1ba4ba4936dbd0b6aa4306f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:14.346026 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa1113-d940-448c-b399-f22c6b9bb12f admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:14.347781 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c1fa1113-d940-448c-b399-f22c6b9bb12f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:14.441824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa1113-d940-448c-b399-f22c6b9bb12f admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:14.525373 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595d68f-67f9-4c83-8860-477dfbbc969e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:14.526539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595d68f-67f9-4c83-8860-477dfbbc969e service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:14.526836 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595d68f-67f9-4c83-8860-477dfbbc969e service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:14.527108 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595d68f-67f9-4c83-8860-477dfbbc969e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:14.529394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-2595d68f-67f9-4c83-8860-477dfbbc969e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:14.606624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595d68f-67f9-4c83-8860-477dfbbc969e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:14.608227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595d68f-67f9-4c83-8860-477dfbbc969e service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:14.618699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1158/2307] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:22:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:22:14.673030 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1012b5-79d1-443f-906e-770e5da415e7 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:14.674638 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-8b1012b5-79d1-443f-906e-770e5da415e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:14.764390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1012b5-79d1-443f-906e-770e5da415e7 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:14.769658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e8762-ce54-4acc-ba98-f1d1f805e18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:14.770420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e8762-ce54-4acc-ba98-f1d1f805e18a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:14.770820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e8762-ce54-4acc-ba98-f1d1f805e18a admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:14.771215 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e8762-ce54-4acc-ba98-f1d1f805e18a admin admin] PATH_INFO: `/v3/projects/36e111682adc40948af455a92fadaa0a` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:14.772684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e8762-ce54-4acc-ba98-f1d1f805e18a admin admin] RBAC: Authorizing `identity:delete_project(project_id=36e111682adc40948af455a92fadaa0a)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:14.774747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e8762-ce54-4acc-ba98-f1d1f805e18a admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:14.830711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-9f8e8762-ce54-4acc-ba98-f1d1f805e18a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36e111682adc40948af455a92fadaa0a', 'request_id': 'req-9f8e8762-ce54-4acc-ba98-f1d1f805e18a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:14.840054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-9f8e8762-ce54-4acc-ba98-f1d1f805e18a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36e111682adc40948af455a92fadaa0a', 'request_id': 'req-9f8e8762-ce54-4acc-ba98-f1d1f805e18a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:14.847055 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-9f8e8762-ce54-4acc-ba98-f1d1f805e18a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36e111682adc40948af455a92fadaa0a', 'request_id': 'req-9f8e8762-ce54-4acc-ba98-f1d1f805e18a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:14.859139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-9f8e8762-ce54-4acc-ba98-f1d1f805e18a admin admin] The token cache is being invalidate because project 36e111682adc40948af455a92fadaa0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:22:14.859427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-9f8e8762-ce54-4acc-ba98-f1d1f805e18a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:14.860221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-9f8e8762-ce54-4acc-ba98-f1d1f805e18a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:14.861774 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1150/2308] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:22:14 2022] DELETE /identity/v3/projects/36e111682adc40948af455a92fadaa0a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:14.868689 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bed85a-4524-4a43-95bc-19ecc75253bf None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:14.869406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bed85a-4524-4a43-95bc-19ecc75253bf None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:14.869406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bed85a-4524-4a43-95bc-19ecc75253bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:14.896670 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-49bed85a-4524-4a43-95bc-19ecc75253bf None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:14.898094 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-49bed85a-4524-4a43-95bc-19ecc75253bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:14.917830 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:14.970010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1159/2309] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:14.971887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8778126-1926-4cea-921e-8b513c352192 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:14.972343 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8778126-1926-4cea-921e-8b513c352192 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:14.972635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8778126-1926-4cea-921e-8b513c352192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:14.994307 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-a8778126-1926-4cea-921e-8b513c352192 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:14.995632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a8778126-1926-4cea-921e-8b513c352192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:15.003569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1151/2310] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:15.007855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bed85a-4524-4a43-95bc-19ecc75253bf None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:15.017403 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bed85a-4524-4a43-95bc-19ecc75253bf None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:15.021993 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09be22-4d69-48dc-8eed-89b223fd8b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:15.022600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09be22-4d69-48dc-8eed-89b223fd8b5e admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:15.022876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09be22-4d69-48dc-8eed-89b223fd8b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:15.023148 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09be22-4d69-48dc-8eed-89b223fd8b5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:15.023845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b09be22-4d69-48dc-8eed-89b223fd8b5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:22:15.024145 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b09be22-4d69-48dc-8eed-89b223fd8b5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:15.025761 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b09be22-4d69-48dc-8eed-89b223fd8b5e admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:15.037963 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1160/2311] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:22:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:22:15.042169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8778126-1926-4cea-921e-8b513c352192 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:15.051517 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8778126-1926-4cea-921e-8b513c352192 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:15.056896 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cae9ff-dd3b-4513-9d3c-e3052d32d78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:15.057503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cae9ff-dd3b-4513-9d3c-e3052d32d78c admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:15.057780 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cae9ff-dd3b-4513-9d3c-e3052d32d78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:15.058050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cae9ff-dd3b-4513-9d3c-e3052d32d78c admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:15.058715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cae9ff-dd3b-4513-9d3c-e3052d32d78c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:15.060600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cae9ff-dd3b-4513-9d3c-e3052d32d78c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:15.088874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1152/2312] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:22:15 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:22:15.093297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09be22-4d69-48dc-8eed-89b223fd8b5e admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:15.104143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09be22-4d69-48dc-8eed-89b223fd8b5e admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:15.162035 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6408c-bb1a-42e8-8723-e91f668cf4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:15.162618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6408c-bb1a-42e8-8723-e91f668cf4f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:15.162894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6408c-bb1a-42e8-8723-e91f668cf4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:15.163164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d6408c-bb1a-42e8-8723-e91f668cf4f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:15.163819 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6408c-bb1a-42e8-8723-e91f668cf4f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:15.165770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d6408c-bb1a-42e8-8723-e91f668cf4f0 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:15.188996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1161/2313] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:22:15 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:22:15.193811 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cae9ff-dd3b-4513-9d3c-e3052d32d78c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:15.199490 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:15.204924 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cae9ff-dd3b-4513-9d3c-e3052d32d78c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:15.215237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8bef4-3f67-4ded-bb85-012b7fb455e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:15.216197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8bef4-3f67-4ded-bb85-012b7fb455e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:15.216474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8bef4-3f67-4ded-bb85-012b7fb455e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:15.216849 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8bef4-3f67-4ded-bb85-012b7fb455e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:15.217547 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8bef4-3f67-4ded-bb85-012b7fb455e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:15.219164 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8bef4-3f67-4ded-bb85-012b7fb455e2 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:15.232951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1153/2314] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:22:15.234483 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6408c-bb1a-42e8-8723-e91f668cf4f0 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:15.246186 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d6408c-bb1a-42e8-8723-e91f668cf4f0 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:15.250959 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06910a4f-7d63-4ab6-838f-1608b16b17a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:15.251568 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06910a4f-7d63-4ab6-838f-1608b16b17a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:15.251879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06910a4f-7d63-4ab6-838f-1608b16b17a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:15.252183 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06910a4f-7d63-4ab6-838f-1608b16b17a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:15.252922 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06910a4f-7d63-4ab6-838f-1608b16b17a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:15.254555 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06910a4f-7d63-4ab6-838f-1608b16b17a9 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:15.268907 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1162/2315] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:22:15.273004 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8bef4-3f67-4ded-bb85-012b7fb455e2 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:15.284585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8bef4-3f67-4ded-bb85-012b7fb455e2 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:15.290240 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca949103-fc15-41c4-9dfa-7d47e8c66a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:15.291268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca949103-fc15-41c4-9dfa-7d47e8c66a1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:15.291714 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca949103-fc15-41c4-9dfa-7d47e8c66a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:15.292139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca949103-fc15-41c4-9dfa-7d47e8c66a1b admin admin] PATH_INFO: `/v3/projects/262a8f0071b645d2a1c60141ab52588c/users/5a7a134ae8354f728145346666b98948/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:15.312083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca949103-fc15-41c4-9dfa-7d47e8c66a1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=262a8f0071b645d2a1c60141ab52588c, user_id=5a7a134ae8354f728145346666b98948, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:15.314880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca949103-fc15-41c4-9dfa-7d47e8c66a1b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:15.331765 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1154/2316] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:22:15 2022] PUT /identity/v3/projects/262a8f0071b645d2a1c60141ab52588c/users/5a7a134ae8354f728145346666b98948/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:15.343854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429123e6-df59-4672-a704-95d4ce038406 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:15.344140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429123e6-df59-4672-a704-95d4ce038406 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:15.344418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429123e6-df59-4672-a704-95d4ce038406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:15.374582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-429123e6-df59-4672-a704-95d4ce038406 None None] MFA Rules not processed for user `5a7a134ae8354f728145346666b98948`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:15.375428 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-429123e6-df59-4672-a704-95d4ce038406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:15.457371 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1163/2317] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:15.475041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca949103-fc15-41c4-9dfa-7d47e8c66a1b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:15.478270 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ca949103-fc15-41c4-9dfa-7d47e8c66a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:15.552544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca949103-fc15-41c4-9dfa-7d47e8c66a1b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:15.558473 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc9e6d-70a8-4ec2-86a8-08dbe4a880d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:15.559285 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc9e6d-70a8-4ec2-86a8-08dbe4a880d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:15.559567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc9e6d-70a8-4ec2-86a8-08dbe4a880d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:15.559861 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc9e6d-70a8-4ec2-86a8-08dbe4a880d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:15.586621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc9e6d-70a8-4ec2-86a8-08dbe4a880d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:15.588245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc9e6d-70a8-4ec2-86a8-08dbe4a880d1 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:15.601003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1155/2318] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:22:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:22:18.217384 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429123e6-df59-4672-a704-95d4ce038406 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:18.236064 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:18.285308 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429123e6-df59-4672-a704-95d4ce038406 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:18.289841 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e01e22-fcc4-4078-ae44-52c0b827c18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:18.290471 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e01e22-fcc4-4078-ae44-52c0b827c18e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:18.290752 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e01e22-fcc4-4078-ae44-52c0b827c18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:18.291026 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e01e22-fcc4-4078-ae44-52c0b827c18e admin admin] PATH_INFO: `/v3/users/5a7a134ae8354f728145346666b98948` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:18.292198 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e01e22-fcc4-4078-ae44-52c0b827c18e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a7a134ae8354f728145346666b98948)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:18.294128 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e01e22-fcc4-4078-ae44-52c0b827c18e admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:18.339700 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-76e01e22-fcc4-4078-ae44-52c0b827c18e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a7a134ae8354f728145346666b98948'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:18.346738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-76e01e22-fcc4-4078-ae44-52c0b827c18e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a7a134ae8354f728145346666b98948'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:18.347401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-76e01e22-fcc4-4078-ae44-52c0b827c18e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a7a134ae8354f728145346666b98948'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:18.360286 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-76e01e22-fcc4-4078-ae44-52c0b827c18e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a7a134ae8354f728145346666b98948'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:18.360971 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-76e01e22-fcc4-4078-ae44-52c0b827c18e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a7a134ae8354f728145346666b98948'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:18.374188 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1164/2319] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:22:18 2022] DELETE /identity/v3/users/5a7a134ae8354f728145346666b98948 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:18.378650 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cdbc3-f2cc-4f4a-82c3-a5c786bd1d63 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:18.379134 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cdbc3-f2cc-4f4a-82c3-a5c786bd1d63 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:18.379717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0cdbc3-f2cc-4f4a-82c3-a5c786bd1d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:18.398265 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-0f0cdbc3-f2cc-4f4a-82c3-a5c786bd1d63 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:18.399459 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-0f0cdbc3-f2cc-4f4a-82c3-a5c786bd1d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:18.459748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1156/2320] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:18.469564 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e01e22-fcc4-4078-ae44-52c0b827c18e admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:18.471021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-76e01e22-fcc4-4078-ae44-52c0b827c18e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:18.529483 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e01e22-fcc4-4078-ae44-52c0b827c18e admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:18.533319 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c389efe9-be58-43ea-a0a4-3de5399c1e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:18.533929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389efe9-be58-43ea-a0a4-3de5399c1e96 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:18.534208 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389efe9-be58-43ea-a0a4-3de5399c1e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:18.534482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c389efe9-be58-43ea-a0a4-3de5399c1e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:18.544688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389efe9-be58-43ea-a0a4-3de5399c1e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:18.546115 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c389efe9-be58-43ea-a0a4-3de5399c1e96 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:18.556664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1165/2321] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:22:18.898772 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0cdbc3-f2cc-4f4a-82c3-a5c786bd1d63 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:18.900290 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-0f0cdbc3-f2cc-4f4a-82c3-a5c786bd1d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:18.914479 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0cdbc3-f2cc-4f4a-82c3-a5c786bd1d63 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:18.920740 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935be90-0780-4efa-a92a-e729d36c5a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:18.921478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935be90-0780-4efa-a92a-e729d36c5a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:18.921891 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935be90-0780-4efa-a92a-e729d36c5a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:18.922148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935be90-0780-4efa-a92a-e729d36c5a6d admin admin] PATH_INFO: `/v3/projects/262a8f0071b645d2a1c60141ab52588c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:18.923327 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935be90-0780-4efa-a92a-e729d36c5a6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=262a8f0071b645d2a1c60141ab52588c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:18.925300 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935be90-0780-4efa-a92a-e729d36c5a6d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:18.972986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0935be90-0780-4efa-a92a-e729d36c5a6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '262a8f0071b645d2a1c60141ab52588c', 'request_id': 'req-0935be90-0780-4efa-a92a-e729d36c5a6d'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:18.981150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0935be90-0780-4efa-a92a-e729d36c5a6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '262a8f0071b645d2a1c60141ab52588c', 'request_id': 'req-0935be90-0780-4efa-a92a-e729d36c5a6d'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:18.986662 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0935be90-0780-4efa-a92a-e729d36c5a6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '262a8f0071b645d2a1c60141ab52588c', 'request_id': 'req-0935be90-0780-4efa-a92a-e729d36c5a6d'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:18.987727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:19.000365 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0935be90-0780-4efa-a92a-e729d36c5a6d admin admin] The token cache is being invalidate because project 262a8f0071b645d2a1c60141ab52588c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:22:19.000668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0935be90-0780-4efa-a92a-e729d36c5a6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:19.001308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-0935be90-0780-4efa-a92a-e729d36c5a6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:19.002814 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1157/2322] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:22:18 2022] DELETE /identity/v3/projects/262a8f0071b645d2a1c60141ab52588c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:19.007882 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232425c-6d4d-4819-8a43-a4025c9fdb6a None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:19.008175 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232425c-6d4d-4819-8a43-a4025c9fdb6a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:19.008454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232425c-6d4d-4819-8a43-a4025c9fdb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:19.029196 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-e232425c-6d4d-4819-8a43-a4025c9fdb6a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:19.030403 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e232425c-6d4d-4819-8a43-a4025c9fdb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:19.063058 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:19.090266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1166/2323] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:19.098398 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29a429-39b5-4a83-b3dd-864f46e3d54e None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:19.099335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29a429-39b5-4a83-b3dd-864f46e3d54e None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:19.099617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29a429-39b5-4a83-b3dd-864f46e3d54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:19.118905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-6a29a429-39b5-4a83-b3dd-864f46e3d54e None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:19.120105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-6a29a429-39b5-4a83-b3dd-864f46e3d54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:19.127969 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1158/2324] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:19.132122 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232425c-6d4d-4819-8a43-a4025c9fdb6a None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:19.141891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232425c-6d4d-4819-8a43-a4025c9fdb6a None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:19.146933 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69920e89-1e99-4982-8826-39a7eef6881b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:19.147578 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69920e89-1e99-4982-8826-39a7eef6881b admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:19.147868 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69920e89-1e99-4982-8826-39a7eef6881b admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:19.148139 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69920e89-1e99-4982-8826-39a7eef6881b admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:19.148869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69920e89-1e99-4982-8826-39a7eef6881b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:22:19.149282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69920e89-1e99-4982-8826-39a7eef6881b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:19.150760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69920e89-1e99-4982-8826-39a7eef6881b admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:19.163015 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1167/2325] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:22:19.167317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29a429-39b5-4a83-b3dd-864f46e3d54e None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:19.176406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29a429-39b5-4a83-b3dd-864f46e3d54e None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:19.181150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5eb7c-5288-4c1e-ad50-4c85a5bf86d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:19.181748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5eb7c-5288-4c1e-ad50-4c85a5bf86d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:19.182030 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5eb7c-5288-4c1e-ad50-4c85a5bf86d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:19.182296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5eb7c-5288-4c1e-ad50-4c85a5bf86d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:19.182945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5eb7c-5288-4c1e-ad50-4c85a5bf86d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:19.184842 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5eb7c-5288-4c1e-ad50-4c85a5bf86d1 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:19.212947 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1159/2326] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:22:19 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:22:19.217513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69920e89-1e99-4982-8826-39a7eef6881b admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:19.227735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69920e89-1e99-4982-8826-39a7eef6881b admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:19.291999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef5176-213c-4909-a3d0-bd46779124a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:19.292625 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef5176-213c-4909-a3d0-bd46779124a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:19.292930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef5176-213c-4909-a3d0-bd46779124a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:19.293208 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef5176-213c-4909-a3d0-bd46779124a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:19.293880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef5176-213c-4909-a3d0-bd46779124a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:19.295803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef5176-213c-4909-a3d0-bd46779124a3 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:19.317067 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1168/2327] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:22:19 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:22:19.321889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5eb7c-5288-4c1e-ad50-4c85a5bf86d1 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:19.332142 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5eb7c-5288-4c1e-ad50-4c85a5bf86d1 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:19.337530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2138ef-3650-481c-9603-5a08d1ddbd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:19.338137 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2138ef-3650-481c-9603-5a08d1ddbd3e admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:19.338419 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2138ef-3650-481c-9603-5a08d1ddbd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:19.338693 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2138ef-3650-481c-9603-5a08d1ddbd3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:19.339385 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2138ef-3650-481c-9603-5a08d1ddbd3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:19.341279 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2138ef-3650-481c-9603-5a08d1ddbd3e admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:19.351118 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1160/2328] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:22:19.355058 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef5176-213c-4909-a3d0-bd46779124a3 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:19.365872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef5176-213c-4909-a3d0-bd46779124a3 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:19.370488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852a257-8eb8-46dc-b735-1aef7722f547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:19.371095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852a257-8eb8-46dc-b735-1aef7722f547 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:19.371374 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852a257-8eb8-46dc-b735-1aef7722f547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:19.371647 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852a257-8eb8-46dc-b735-1aef7722f547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:19.372340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852a257-8eb8-46dc-b735-1aef7722f547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:19.373965 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852a257-8eb8-46dc-b735-1aef7722f547 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:19.385826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1169/2329] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:22:19.390172 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2138ef-3650-481c-9603-5a08d1ddbd3e admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:19.400584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2138ef-3650-481c-9603-5a08d1ddbd3e admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:19.406064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcb6ad-b858-44ea-9279-9b5c902ce9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:19.406736 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcb6ad-b858-44ea-9279-9b5c902ce9be admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:19.407020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcb6ad-b858-44ea-9279-9b5c902ce9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:19.407300 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcb6ad-b858-44ea-9279-9b5c902ce9be admin admin] PATH_INFO: `/v3/projects/9b1c415a83d34029a1c2ef13835c3df9/users/53fecf0a11114717a0bc474172d5846b/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:19.423034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcb6ad-b858-44ea-9279-9b5c902ce9be admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b1c415a83d34029a1c2ef13835c3df9, user_id=53fecf0a11114717a0bc474172d5846b, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:19.426300 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcb6ad-b858-44ea-9279-9b5c902ce9be admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:19.440326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1161/2330] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:22:19 2022] PUT /identity/v3/projects/9b1c415a83d34029a1c2ef13835c3df9/users/53fecf0a11114717a0bc474172d5846b/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:19.450721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e9561-6531-40a3-bf0f-50a65daa667e None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:19.451011 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e9561-6531-40a3-bf0f-50a65daa667e None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:19.451279 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e9561-6531-40a3-bf0f-50a65daa667e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:19.479353 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-767e9561-6531-40a3-bf0f-50a65daa667e None None] MFA Rules not processed for user `53fecf0a11114717a0bc474172d5846b`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:19.480527 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-767e9561-6531-40a3-bf0f-50a65daa667e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:19.550391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1170/2331] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:19.562626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcb6ad-b858-44ea-9279-9b5c902ce9be admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:19.564404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-08fcb6ad-b858-44ea-9279-9b5c902ce9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:19.629044 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcb6ad-b858-44ea-9279-9b5c902ce9be admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:19.634265 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a5568-e15b-4557-a6bf-7feb422b437a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ff31d52ecf4c7d93a65bbfca7285c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:19.634896 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a5568-e15b-4557-a6bf-7feb422b437a service glance] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:19.635176 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a5568-e15b-4557-a6bf-7feb422b437a service glance] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:19.635498 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a5568-e15b-4557-a6bf-7feb422b437a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:19.644429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a5568-e15b-4557-a6bf-7feb422b437a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:19.645972 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578a5568-e15b-4557-a6bf-7feb422b437a service glance] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:19.655144 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1162/2332] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:22:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:22:22.771708 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320020dc-6ff0-421f-9d09-59479a813ba5 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:22.772438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320020dc-6ff0-421f-9d09-59479a813ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:22.772651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320020dc-6ff0-421f-9d09-59479a813ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:22.797300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-320020dc-6ff0-421f-9d09-59479a813ba5 None None] MFA Rules not processed for user `e54e713b52e84393b018807f8153922e`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:22.798551 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-320020dc-6ff0-421f-9d09-59479a813ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:22.831977 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:22.859998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1171/2333] 10.222.0.45 () {58 vars in 1009 bytes} [Thu Aug 4 02:22:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:22.908274 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a5568-e15b-4557-a6bf-7feb422b437a service glance] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:22.909762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-578a5568-e15b-4557-a6bf-7feb422b437a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:22.983161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a5568-e15b-4557-a6bf-7feb422b437a service glance] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:22.988101 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575dc0e5-d5c4-49a4-af28-25e409449481 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '155a7276efab426cb34175683c475f33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:22.988752 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dc0e5-d5c4-49a4-af28-25e409449481 service swift] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:22.989036 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dc0e5-d5c4-49a4-af28-25e409449481 service swift] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:22.989308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575dc0e5-d5c4-49a4-af28-25e409449481 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:22.998243 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575dc0e5-d5c4-49a4-af28-25e409449481 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:22.999861 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575dc0e5-d5c4-49a4-af28-25e409449481 service swift] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:23.009364 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1163/2334] 10.222.0.45 () {60 vars in 1384 bytes} [Thu Aug 4 02:22:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 02:22:32.342113 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320020dc-6ff0-421f-9d09-59479a813ba5 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:32.343442 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-320020dc-6ff0-421f-9d09-59479a813ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:32.399957 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320020dc-6ff0-421f-9d09-59479a813ba5 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:32.405192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62eb72f-20a4-47f2-9cbf-8c90572542e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:32.405823 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62eb72f-20a4-47f2-9cbf-8c90572542e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:32.406105 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62eb72f-20a4-47f2-9cbf-8c90572542e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:32.406371 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62eb72f-20a4-47f2-9cbf-8c90572542e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:32.408161 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f62eb72f-20a4-47f2-9cbf-8c90572542e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:32.418791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62eb72f-20a4-47f2-9cbf-8c90572542e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:32.420219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62eb72f-20a4-47f2-9cbf-8c90572542e5 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:32.429531 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1172/2335] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:22:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:22:33.005292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=88,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:34.850156 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575dc0e5-d5c4-49a4-af28-25e409449481 service swift] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:34.851517 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-575dc0e5-d5c4-49a4-af28-25e409449481 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:34.930457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575dc0e5-d5c4-49a4-af28-25e409449481 service swift] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:34.936295 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85916186-b115-4e47-b45d-11c315363622 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:34.936952 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85916186-b115-4e47-b45d-11c315363622 service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:34.937234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85916186-b115-4e47-b45d-11c315363622 service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:34.937537 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85916186-b115-4e47-b45d-11c315363622 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:34.939319 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-85916186-b115-4e47-b45d-11c315363622 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:35.012341 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85916186-b115-4e47-b45d-11c315363622 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:35.013977 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85916186-b115-4e47-b45d-11c315363622 service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:35.023221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1164/2336] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:22:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:22:36.613569 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62eb72f-20a4-47f2-9cbf-8c90572542e5 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:36.615215 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-f62eb72f-20a4-47f2-9cbf-8c90572542e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:36.698498 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62eb72f-20a4-47f2-9cbf-8c90572542e5 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:36.703839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347ecd4-f411-423b-a185-91a345ef03a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796772cf491047c8b0a609dae9402307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:36.704464 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347ecd4-f411-423b-a185-91a345ef03a6 service placement] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:36.704780 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347ecd4-f411-423b-a185-91a345ef03a6 service placement] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:36.705103 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347ecd4-f411-423b-a185-91a345ef03a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:36.707124 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7347ecd4-f411-423b-a185-91a345ef03a6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:36.720763 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347ecd4-f411-423b-a185-91a345ef03a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:36.722969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347ecd4-f411-423b-a185-91a345ef03a6 service placement] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:36.737253 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1173/2337] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:22:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:22:38.356226 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373d759-72f8-4953-989e-dc2914aff349 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:38.356785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373d759-72f8-4953-989e-dc2914aff349 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:38.356785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2373d759-72f8-4953-989e-dc2914aff349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:38.376272 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-2373d759-72f8-4953-989e-dc2914aff349 None None] MFA Rules not processed for user `e54e713b52e84393b018807f8153922e`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:38.377475 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2373d759-72f8-4953-989e-dc2914aff349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:38.385330 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1165/2338] 10.222.0.45 () {58 vars in 1009 bytes} [Thu Aug 4 02:22:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:22:38.397306 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347ecd4-f411-423b-a185-91a345ef03a6 service placement] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:38.410065 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347ecd4-f411-423b-a185-91a345ef03a6 service placement] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:38.415183 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d585205-71ea-4ae6-83bb-55c7c9f76b88 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '155a7276efab426cb34175683c475f33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:38.415852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d585205-71ea-4ae6-83bb-55c7c9f76b88 service swift] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:38.416133 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d585205-71ea-4ae6-83bb-55c7c9f76b88 service swift] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:38.416412 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d585205-71ea-4ae6-83bb-55c7c9f76b88 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:38.434224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d585205-71ea-4ae6-83bb-55c7c9f76b88 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:38.435840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d585205-71ea-4ae6-83bb-55c7c9f76b88 service swift] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:38.444537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1174/2339] 10.222.0.45 () {60 vars in 1384 bytes} [Thu Aug 4 02:22:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 02:22:47.833173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373d759-72f8-4953-989e-dc2914aff349 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:47.903820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2373d759-72f8-4953-989e-dc2914aff349 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:47.908368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a464e3-7710-479a-8492-7ba4ee48ace8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:47.909036 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a464e3-7710-479a-8492-7ba4ee48ace8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:47.909318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a464e3-7710-479a-8492-7ba4ee48ace8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:47.909593 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a464e3-7710-479a-8492-7ba4ee48ace8 admin admin] PATH_INFO: `/v3/users/53fecf0a11114717a0bc474172d5846b` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:47.910759 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a464e3-7710-479a-8492-7ba4ee48ace8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53fecf0a11114717a0bc474172d5846b)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:47.912703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a464e3-7710-479a-8492-7ba4ee48ace8 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:47.949485 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:47.962782 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d3a464e3-7710-479a-8492-7ba4ee48ace8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53fecf0a11114717a0bc474172d5846b'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:47.963971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d3a464e3-7710-479a-8492-7ba4ee48ace8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53fecf0a11114717a0bc474172d5846b'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:47.964602 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d3a464e3-7710-479a-8492-7ba4ee48ace8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53fecf0a11114717a0bc474172d5846b'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:47.976645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d3a464e3-7710-479a-8492-7ba4ee48ace8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53fecf0a11114717a0bc474172d5846b'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:47.983407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d3a464e3-7710-479a-8492-7ba4ee48ace8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53fecf0a11114717a0bc474172d5846b'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:47.997636 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1166/2340] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:22:47 2022] DELETE /identity/v3/users/53fecf0a11114717a0bc474172d5846b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:48.002229 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b071b-9eff-405c-84e1-d5de2fb076b6 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:48.002751 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b071b-9eff-405c-84e1-d5de2fb076b6 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:48.002751 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36b071b-9eff-405c-84e1-d5de2fb076b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:48.024787 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-e36b071b-9eff-405c-84e1-d5de2fb076b6 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:48.026764 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e36b071b-9eff-405c-84e1-d5de2fb076b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:48.077828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:48.087154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1175/2341] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:48.097729 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a464e3-7710-479a-8492-7ba4ee48ace8 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:48.099135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d3a464e3-7710-479a-8492-7ba4ee48ace8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:48.156160 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a464e3-7710-479a-8492-7ba4ee48ace8 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:48.159841 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd14aab-57dd-4663-a6ca-19e1a321f975 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:48.160463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd14aab-57dd-4663-a6ca-19e1a321f975 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:48.160774 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd14aab-57dd-4663-a6ca-19e1a321f975 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:48.161051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd14aab-57dd-4663-a6ca-19e1a321f975 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:48.170153 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd14aab-57dd-4663-a6ca-19e1a321f975 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:48.171579 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd14aab-57dd-4663-a6ca-19e1a321f975 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:48.180896 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1167/2342] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:22:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:22:48.465200 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b071b-9eff-405c-84e1-d5de2fb076b6 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:48.466949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e36b071b-9eff-405c-84e1-d5de2fb076b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:48.483049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36b071b-9eff-405c-84e1-d5de2fb076b6 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:48.487945 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2826b-68bc-40b5-91ca-f36363da0bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:48.488664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2826b-68bc-40b5-91ca-f36363da0bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:48.488946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2826b-68bc-40b5-91ca-f36363da0bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:48.489217 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2826b-68bc-40b5-91ca-f36363da0bc0 admin admin] PATH_INFO: `/v3/projects/9b1c415a83d34029a1c2ef13835c3df9` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:48.490375 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2826b-68bc-40b5-91ca-f36363da0bc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b1c415a83d34029a1c2ef13835c3df9)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:48.492291 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2826b-68bc-40b5-91ca-f36363da0bc0 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:48.545623 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5be2826b-68bc-40b5-91ca-f36363da0bc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b1c415a83d34029a1c2ef13835c3df9', 'request_id': 'req-5be2826b-68bc-40b5-91ca-f36363da0bc0'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:48.554050 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5be2826b-68bc-40b5-91ca-f36363da0bc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b1c415a83d34029a1c2ef13835c3df9', 'request_id': 'req-5be2826b-68bc-40b5-91ca-f36363da0bc0'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:48.560916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5be2826b-68bc-40b5-91ca-f36363da0bc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b1c415a83d34029a1c2ef13835c3df9', 'request_id': 'req-5be2826b-68bc-40b5-91ca-f36363da0bc0'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:48.572509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5be2826b-68bc-40b5-91ca-f36363da0bc0 admin admin] The token cache is being invalidate because project 9b1c415a83d34029a1c2ef13835c3df9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:22:48.573382 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5be2826b-68bc-40b5-91ca-f36363da0bc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:48.574039 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5be2826b-68bc-40b5-91ca-f36363da0bc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:22:48.575599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1176/2343] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:22:48 2022] DELETE /identity/v3/projects/9b1c415a83d34029a1c2ef13835c3df9 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:22:48.580581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a648df-3b94-4c9f-bf76-572787a652be None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:48.580912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a648df-3b94-4c9f-bf76-572787a652be None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:48.581263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a648df-3b94-4c9f-bf76-572787a652be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:48.601657 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-c5a648df-3b94-4c9f-bf76-572787a652be None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:48.602957 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-c5a648df-3b94-4c9f-bf76-572787a652be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:48.665527 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1168/2344] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:48.673852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53cdf3-fb8c-42d5-a142-154b571ff1eb None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:48.674130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53cdf3-fb8c-42d5-a142-154b571ff1eb None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:48.674396 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f53cdf3-fb8c-42d5-a142-154b571ff1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:48.694056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-0f53cdf3-fb8c-42d5-a142-154b571ff1eb None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:48.695236 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-0f53cdf3-fb8c-42d5-a142-154b571ff1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:48.703006 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1177/2345] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:48.707222 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a648df-3b94-4c9f-bf76-572787a652be None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:48.717587 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a648df-3b94-4c9f-bf76-572787a652be None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:48.722178 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc8754-2820-49ab-9050-8b49de2aa2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:48.722811 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc8754-2820-49ab-9050-8b49de2aa2f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:48.723089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc8754-2820-49ab-9050-8b49de2aa2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:48.723361 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc8754-2820-49ab-9050-8b49de2aa2f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:48.724055 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc8754-2820-49ab-9050-8b49de2aa2f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:22:48.724354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc8754-2820-49ab-9050-8b49de2aa2f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:48.725981 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc8754-2820-49ab-9050-8b49de2aa2f9 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:48.739344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1169/2346] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:22:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:22:48.744651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53cdf3-fb8c-42d5-a142-154b571ff1eb None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:48.754747 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f53cdf3-fb8c-42d5-a142-154b571ff1eb None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:48.759448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bd828-24c0-4f52-9b52-0a696563a6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:48.760056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bd828-24c0-4f52-9b52-0a696563a6d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:48.760371 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bd828-24c0-4f52-9b52-0a696563a6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:48.760673 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bd828-24c0-4f52-9b52-0a696563a6d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:48.761338 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bd828-24c0-4f52-9b52-0a696563a6d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:48.763223 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bd828-24c0-4f52-9b52-0a696563a6d4 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:48.789824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1178/2347] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:22:48 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:22:48.794589 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc8754-2820-49ab-9050-8b49de2aa2f9 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:48.804457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc8754-2820-49ab-9050-8b49de2aa2f9 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:48.844171 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:48.864444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c220b-fcaa-4334-990e-2fdefd78debf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:48.865396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c220b-fcaa-4334-990e-2fdefd78debf admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:48.865678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c220b-fcaa-4334-990e-2fdefd78debf admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:48.865947 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c220b-fcaa-4334-990e-2fdefd78debf admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:48.866609 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c220b-fcaa-4334-990e-2fdefd78debf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:48.868519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c220b-fcaa-4334-990e-2fdefd78debf admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:48.889468 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1170/2348] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:22:48 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:22:48.893751 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bd828-24c0-4f52-9b52-0a696563a6d4 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:48.903910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bd828-24c0-4f52-9b52-0a696563a6d4 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:48.908458 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9c776-1766-4646-b375-38463af7416a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:48.909080 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9c776-1766-4646-b375-38463af7416a admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:48.909355 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9c776-1766-4646-b375-38463af7416a admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:48.909620 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9c776-1766-4646-b375-38463af7416a admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:48.910296 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9c776-1766-4646-b375-38463af7416a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:48.911897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9c776-1766-4646-b375-38463af7416a admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:48.922654 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1179/2349] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:22:48.927685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c220b-fcaa-4334-990e-2fdefd78debf admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:48.938447 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c220b-fcaa-4334-990e-2fdefd78debf admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:48.943020 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcba7087-0efc-4a77-af3e-e71e8e357997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:48.943618 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba7087-0efc-4a77-af3e-e71e8e357997 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:48.943896 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba7087-0efc-4a77-af3e-e71e8e357997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:48.944169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcba7087-0efc-4a77-af3e-e71e8e357997 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:48.944888 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba7087-0efc-4a77-af3e-e71e8e357997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:48.946491 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcba7087-0efc-4a77-af3e-e71e8e357997 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:48.957183 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1171/2350] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:22:48.961362 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9c776-1766-4646-b375-38463af7416a admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:48.971618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9c776-1766-4646-b375-38463af7416a admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:48.976246 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6a900-4b4c-4d09-bdc7-5cc7b0f720df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:48.976936 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6a900-4b4c-4d09-bdc7-5cc7b0f720df admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:48.977247 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6a900-4b4c-4d09-bdc7-5cc7b0f720df admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:48.977525 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a6a900-4b4c-4d09-bdc7-5cc7b0f720df admin admin] PATH_INFO: `/v3/projects/60356a5ecc7648f5998f782087e3a437/users/3ab5e56697eb4650ad82b9eb44ed6dd2/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:48.992417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6a900-4b4c-4d09-bdc7-5cc7b0f720df admin admin] RBAC: Authorizing `identity:create_grant(project_id=60356a5ecc7648f5998f782087e3a437, user_id=3ab5e56697eb4650ad82b9eb44ed6dd2, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:48.995794 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a6a900-4b4c-4d09-bdc7-5cc7b0f720df admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:49.008594 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1180/2351] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:22:48 2022] PUT /identity/v3/projects/60356a5ecc7648f5998f782087e3a437/users/3ab5e56697eb4650ad82b9eb44ed6dd2/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:22:49.018990 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec290b-907d-4f78-b59c-a4e27e8b357a None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:49.019273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec290b-907d-4f78-b59c-a4e27e8b357a None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:49.019543 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ec290b-907d-4f78-b59c-a4e27e8b357a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:49.046835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-78ec290b-907d-4f78-b59c-a4e27e8b357a None None] MFA Rules not processed for user `3ab5e56697eb4650ad82b9eb44ed6dd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:22:49.048040 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-78ec290b-907d-4f78-b59c-a4e27e8b357a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:49.112104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1172/2352] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:22:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:22:49.123138 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6a900-4b4c-4d09-bdc7-5cc7b0f720df admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:22:49.124868 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a3a6a900-4b4c-4d09-bdc7-5cc7b0f720df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:22:49.180913 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a6a900-4b4c-4d09-bdc7-5cc7b0f720df admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:22:49.184532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a9f9f-6311-4e30-a95f-fdd251469e0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:22:49.185168 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a9f9f-6311-4e30-a95f-fdd251469e0c service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:22:49.185445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a9f9f-6311-4e30-a95f-fdd251469e0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:22:49.185713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a9f9f-6311-4e30-a95f-fdd251469e0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:22:49.193819 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a9f9f-6311-4e30-a95f-fdd251469e0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:22:49.195230 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a9f9f-6311-4e30-a95f-fdd251469e0c service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:22:49.203759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1181/2353] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:22:49 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:22:59.109175 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=48,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:22:59.199848 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:06.557627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec290b-907d-4f78-b59c-a4e27e8b357a None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:06.561437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-78ec290b-907d-4f78-b59c-a4e27e8b357a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:06.647882 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ec290b-907d-4f78-b59c-a4e27e8b357a None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:06.654906 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac0139-2eb6-4225-b5ad-aea815729431 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:06.655684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac0139-2eb6-4225-b5ad-aea815729431 service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:06.656061 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac0139-2eb6-4225-b5ad-aea815729431 service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:06.656410 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac0139-2eb6-4225-b5ad-aea815729431 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:06.666519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e0ac0139-2eb6-4225-b5ad-aea815729431 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:06.681835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac0139-2eb6-4225-b5ad-aea815729431 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:06.683427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac0139-2eb6-4225-b5ad-aea815729431 service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:06.695787 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1173/2354] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:23:08.184470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60669b-7a79-415d-b37e-c3232f0d49ac None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:08.185551 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60669b-7a79-415d-b37e-c3232f0d49ac None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:08.185847 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca60669b-7a79-415d-b37e-c3232f0d49ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:08.206142 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-ca60669b-7a79-415d-b37e-c3232f0d49ac None None] MFA Rules not processed for user `e54e713b52e84393b018807f8153922e`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:08.207352 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-ca60669b-7a79-415d-b37e-c3232f0d49ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:08.276429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1182/2355] 10.222.0.45 () {58 vars in 967 bytes} [Thu Aug 4 02:23:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:23:08.289167 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac0139-2eb6-4225-b5ad-aea815729431 service nova] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:08.290804 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e0ac0139-2eb6-4225-b5ad-aea815729431 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:08.360849 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac0139-2eb6-4225-b5ad-aea815729431 service nova] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:08.365831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9f2a1-0ec6-404f-8714-242542a197c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '155a7276efab426cb34175683c475f33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:08.366457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9f2a1-0ec6-404f-8714-242542a197c3 service swift] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:08.366738 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9f2a1-0ec6-404f-8714-242542a197c3 service swift] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:08.367011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9f2a1-0ec6-404f-8714-242542a197c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:08.375694 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9f2a1-0ec6-404f-8714-242542a197c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:08.377350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9f2a1-0ec6-404f-8714-242542a197c3 service swift] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:08.385262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1174/2356] 10.222.0.45 () {60 vars in 1384 bytes} [Thu Aug 4 02:23:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 04 02:23:18.269165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=25 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:18.382061 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=66 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:20.744011 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60669b-7a79-415d-b37e-c3232f0d49ac None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:20.745361 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-ca60669b-7a79-415d-b37e-c3232f0d49ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:20.760909 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca60669b-7a79-415d-b37e-c3232f0d49ac None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:20.766250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647caaa5-0d65-4b2b-91ff-e2ad2e39fac2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:20.766876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647caaa5-0d65-4b2b-91ff-e2ad2e39fac2 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:20.767157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647caaa5-0d65-4b2b-91ff-e2ad2e39fac2 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:20.767434 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647caaa5-0d65-4b2b-91ff-e2ad2e39fac2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:20.769180 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-647caaa5-0d65-4b2b-91ff-e2ad2e39fac2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:20.782227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647caaa5-0d65-4b2b-91ff-e2ad2e39fac2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:20.783845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647caaa5-0d65-4b2b-91ff-e2ad2e39fac2 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:20.799733 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1183/2357] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:22.592367 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9f2a1-0ec6-404f-8714-242542a197c3 service swift] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:22.594808 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-95b9f2a1-0ec6-404f-8714-242542a197c3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:22.656658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9f2a1-0ec6-404f-8714-242542a197c3 service swift] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:22.661205 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b9819-f36e-4be1-9720-f8a74bfc14ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:22.662041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b9819-f36e-4be1-9720-f8a74bfc14ef service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:22.663270 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b9819-f36e-4be1-9720-f8a74bfc14ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:22.663866 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b9819-f36e-4be1-9720-f8a74bfc14ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:22.665937 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-981b9819-f36e-4be1-9720-f8a74bfc14ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:22.751532 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b9819-f36e-4be1-9720-f8a74bfc14ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:22.753441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b9819-f36e-4be1-9720-f8a74bfc14ef service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:22.771828 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1175/2358] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:22 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:24.180176 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647caaa5-0d65-4b2b-91ff-e2ad2e39fac2 service nova] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:24.194015 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647caaa5-0d65-4b2b-91ff-e2ad2e39fac2 service nova] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:24.205725 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cf2dbf-a961-49d2-886b-b736b1fddafe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:24.206608 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf2dbf-a961-49d2-886b-b736b1fddafe service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:24.207045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf2dbf-a961-49d2-886b-b736b1fddafe service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:24.207497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cf2dbf-a961-49d2-886b-b736b1fddafe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:24.210194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c3cf2dbf-a961-49d2-886b-b736b1fddafe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:24.254207 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf2dbf-a961-49d2-886b-b736b1fddafe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:24.255831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cf2dbf-a961-49d2-886b-b736b1fddafe service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:24.274062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1184/2359] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:26.874925 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b9819-f36e-4be1-9720-f8a74bfc14ef service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:26.951275 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b9819-f36e-4be1-9720-f8a74bfc14ef service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:26.956508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ebcdc-8d91-4429-9da0-c06e44611689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:26.957150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ebcdc-8d91-4429-9da0-c06e44611689 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:26.957431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ebcdc-8d91-4429-9da0-c06e44611689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:26.957748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ebcdc-8d91-4429-9da0-c06e44611689 admin admin] PATH_INFO: `/v3/users/3ab5e56697eb4650ad82b9eb44ed6dd2` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:26.958916 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ebcdc-8d91-4429-9da0-c06e44611689 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ab5e56697eb4650ad82b9eb44ed6dd2)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:26.960940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ebcdc-8d91-4429-9da0-c06e44611689 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:27.012114 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d62ebcdc-8d91-4429-9da0-c06e44611689 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ab5e56697eb4650ad82b9eb44ed6dd2'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:27.012719 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d62ebcdc-8d91-4429-9da0-c06e44611689 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ab5e56697eb4650ad82b9eb44ed6dd2'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:27.013461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d62ebcdc-8d91-4429-9da0-c06e44611689 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ab5e56697eb4650ad82b9eb44ed6dd2'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:27.021514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:27.029195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d62ebcdc-8d91-4429-9da0-c06e44611689 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ab5e56697eb4650ad82b9eb44ed6dd2'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:27.038353 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d62ebcdc-8d91-4429-9da0-c06e44611689 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ab5e56697eb4650ad82b9eb44ed6dd2'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:27.051263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1176/2360] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:23:26 2022] DELETE /identity/v3/users/3ab5e56697eb4650ad82b9eb44ed6dd2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:27.055770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650fd4e-f01a-49b4-9733-46f014e1e80b None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:27.056280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650fd4e-f01a-49b4-9733-46f014e1e80b None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:27.056380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650fd4e-f01a-49b4-9733-46f014e1e80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:27.074858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-2650fd4e-f01a-49b4-9733-46f014e1e80b None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:27.076122 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-2650fd4e-f01a-49b4-9733-46f014e1e80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:27.141670 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1185/2361] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:27.151477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ebcdc-8d91-4429-9da0-c06e44611689 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:27.152951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d62ebcdc-8d91-4429-9da0-c06e44611689 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:27.219834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ebcdc-8d91-4429-9da0-c06e44611689 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:27.223539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26866803-0d5f-4b5a-ad64-964fff5d2612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:27.224149 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26866803-0d5f-4b5a-ad64-964fff5d2612 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:27.224446 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26866803-0d5f-4b5a-ad64-964fff5d2612 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:27.224720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26866803-0d5f-4b5a-ad64-964fff5d2612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:27.234915 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26866803-0d5f-4b5a-ad64-964fff5d2612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:27.236340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26866803-0d5f-4b5a-ad64-964fff5d2612 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:27.246964 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1177/2362] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:27.544812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650fd4e-f01a-49b4-9733-46f014e1e80b None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:27.544812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-2650fd4e-f01a-49b4-9733-46f014e1e80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:27.569104 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650fd4e-f01a-49b4-9733-46f014e1e80b None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:27.570391 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26866803-0d5f-4b5a-ad64-964fff5d2612 service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:27.574816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e645b50f-9a5e-4a11-b5f7-9a38b6b4ed29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:27.575533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645b50f-9a5e-4a11-b5f7-9a38b6b4ed29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:27.576047 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645b50f-9a5e-4a11-b5f7-9a38b6b4ed29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:27.576722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e645b50f-9a5e-4a11-b5f7-9a38b6b4ed29 admin admin] PATH_INFO: `/v3/projects/60356a5ecc7648f5998f782087e3a437` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:27.578524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645b50f-9a5e-4a11-b5f7-9a38b6b4ed29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60356a5ecc7648f5998f782087e3a437)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:27.579808 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26866803-0d5f-4b5a-ad64-964fff5d2612 service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:27.580966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e645b50f-9a5e-4a11-b5f7-9a38b6b4ed29 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:27.587377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09a48d-f8e3-4502-9c50-1809d904e9fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:27.588074 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09a48d-f8e3-4502-9c50-1809d904e9fa service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:27.588417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09a48d-f8e3-4502-9c50-1809d904e9fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:27.588824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09a48d-f8e3-4502-9c50-1809d904e9fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:27.590984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ee09a48d-f8e3-4502-9c50-1809d904e9fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:27.607753 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09a48d-f8e3-4502-9c50-1809d904e9fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:27.609865 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09a48d-f8e3-4502-9c50-1809d904e9fa service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:27.620753 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1178/2363] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:23:27.665040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-e645b50f-9a5e-4a11-b5f7-9a38b6b4ed29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60356a5ecc7648f5998f782087e3a437', 'request_id': 'req-e645b50f-9a5e-4a11-b5f7-9a38b6b4ed29'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:27.676120 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-e645b50f-9a5e-4a11-b5f7-9a38b6b4ed29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60356a5ecc7648f5998f782087e3a437', 'request_id': 'req-e645b50f-9a5e-4a11-b5f7-9a38b6b4ed29'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:27.692857 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-e645b50f-9a5e-4a11-b5f7-9a38b6b4ed29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60356a5ecc7648f5998f782087e3a437', 'request_id': 'req-e645b50f-9a5e-4a11-b5f7-9a38b6b4ed29'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:27.698582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-e645b50f-9a5e-4a11-b5f7-9a38b6b4ed29 admin admin] The token cache is being invalidate because project 60356a5ecc7648f5998f782087e3a437 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:23:27.699005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-e645b50f-9a5e-4a11-b5f7-9a38b6b4ed29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:27.699696 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-e645b50f-9a5e-4a11-b5f7-9a38b6b4ed29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:27.701342 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1186/2364] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:23:27 2022] DELETE /identity/v3/projects/60356a5ecc7648f5998f782087e3a437 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:27.707946 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebeb53-bfff-40f8-bf10-0a0c751f1da0 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:27.708345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebeb53-bfff-40f8-bf10-0a0c751f1da0 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:27.709059 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bebeb53-bfff-40f8-bf10-0a0c751f1da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:27.731256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-8bebeb53-bfff-40f8-bf10-0a0c751f1da0 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:27.732575 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-8bebeb53-bfff-40f8-bf10-0a0c751f1da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:27.796919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1179/2365] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:27.805057 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950d18f-9367-44c2-9c79-40bcc08f668c None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:27.805349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950d18f-9367-44c2-9c79-40bcc08f668c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:27.805664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2950d18f-9367-44c2-9c79-40bcc08f668c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:27.831737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-2950d18f-9367-44c2-9c79-40bcc08f668c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:27.833263 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-2950d18f-9367-44c2-9c79-40bcc08f668c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:27.841979 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1187/2366] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:23:27.846091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebeb53-bfff-40f8-bf10-0a0c751f1da0 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:27.857169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bebeb53-bfff-40f8-bf10-0a0c751f1da0 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:27.863825 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb24bd4-ce07-4280-ab23-d753a58d6cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:27.864447 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb24bd4-ce07-4280-ab23-d753a58d6cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:27.864757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb24bd4-ce07-4280-ab23-d753a58d6cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:27.865032 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb24bd4-ce07-4280-ab23-d753a58d6cb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:27.865726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb24bd4-ce07-4280-ab23-d753a58d6cb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:23:27.866029 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb24bd4-ce07-4280-ab23-d753a58d6cb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:27.867616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb24bd4-ce07-4280-ab23-d753a58d6cb6 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:27.883755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1180/2367] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:23:27.884909 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950d18f-9367-44c2-9c79-40bcc08f668c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:27.895196 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2950d18f-9367-44c2-9c79-40bcc08f668c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:27.899921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c620c1-e6ae-4910-ab85-4cac7028062f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:27.900546 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c620c1-e6ae-4910-ab85-4cac7028062f admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:27.900858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c620c1-e6ae-4910-ab85-4cac7028062f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:27.901133 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c620c1-e6ae-4910-ab85-4cac7028062f admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:27.901814 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c620c1-e6ae-4910-ab85-4cac7028062f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:27.903703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c620c1-e6ae-4910-ab85-4cac7028062f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:27.938199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1188/2368] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:23:27 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:23:27.939384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb24bd4-ce07-4280-ab23-d753a58d6cb6 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:27.949503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb24bd4-ce07-4280-ab23-d753a58d6cb6 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:27.998168 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:28.020176 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e36cc-03e3-4a53-806a-c6f850048ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:28.020872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e36cc-03e3-4a53-806a-c6f850048ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:28.021761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e36cc-03e3-4a53-806a-c6f850048ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:28.021964 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e36cc-03e3-4a53-806a-c6f850048ca1 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:28.022741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e36cc-03e3-4a53-806a-c6f850048ca1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:28.026532 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e36cc-03e3-4a53-806a-c6f850048ca1 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:28.065267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1181/2369] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:23:27 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:28.065975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c620c1-e6ae-4910-ab85-4cac7028062f admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:28.076883 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c620c1-e6ae-4910-ab85-4cac7028062f admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:28.081591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e300dd-07c4-4e80-9388-c2731f21c663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:28.082213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e300dd-07c4-4e80-9388-c2731f21c663 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:28.082490 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e300dd-07c4-4e80-9388-c2731f21c663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:28.082964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e300dd-07c4-4e80-9388-c2731f21c663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:28.083723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e300dd-07c4-4e80-9388-c2731f21c663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:28.085465 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e300dd-07c4-4e80-9388-c2731f21c663 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:28.097473 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1189/2370] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:28.101429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e36cc-03e3-4a53-806a-c6f850048ca1 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:28.114475 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e36cc-03e3-4a53-806a-c6f850048ca1 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:28.121296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f512f4-3e44-4bc5-aca3-69b9f541fea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:28.121999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f512f4-3e44-4bc5-aca3-69b9f541fea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:28.122392 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f512f4-3e44-4bc5-aca3-69b9f541fea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:28.122739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f512f4-3e44-4bc5-aca3-69b9f541fea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:28.124197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f512f4-3e44-4bc5-aca3-69b9f541fea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:28.126616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f512f4-3e44-4bc5-aca3-69b9f541fea6 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:28.140533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1182/2371] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:28.144729 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e300dd-07c4-4e80-9388-c2731f21c663 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:28.159429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e300dd-07c4-4e80-9388-c2731f21c663 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:28.165208 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e8c1c-3adb-4661-b079-30a573ee461e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:28.165885 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e8c1c-3adb-4661-b079-30a573ee461e admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:28.166356 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e8c1c-3adb-4661-b079-30a573ee461e admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:28.166658 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e8c1c-3adb-4661-b079-30a573ee461e admin admin] PATH_INFO: `/v3/projects/19eead1456de4c6b9a2a0b0524674f0f/users/937131b7fd0048bf97cc2d0b5c3504f2/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:28.187766 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e8c1c-3adb-4661-b079-30a573ee461e admin admin] RBAC: Authorizing `identity:create_grant(project_id=19eead1456de4c6b9a2a0b0524674f0f, user_id=937131b7fd0048bf97cc2d0b5c3504f2, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:28.190833 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e8c1c-3adb-4661-b079-30a573ee461e admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:28.194165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:28.209378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1190/2372] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:28 2022] PUT /identity/v3/projects/19eead1456de4c6b9a2a0b0524674f0f/users/937131b7fd0048bf97cc2d0b5c3504f2/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:28.221146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f38efb-5027-486f-a6fb-ef7da97cf0a6 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:28.221433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f38efb-5027-486f-a6fb-ef7da97cf0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:28.221705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f38efb-5027-486f-a6fb-ef7da97cf0a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:28.257536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-24f38efb-5027-486f-a6fb-ef7da97cf0a6 None None] MFA Rules not processed for user `937131b7fd0048bf97cc2d0b5c3504f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:28.258675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-24f38efb-5027-486f-a6fb-ef7da97cf0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:28.269066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e8c1c-3adb-4661-b079-30a573ee461e admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:28.271366 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7a5e8c1c-3adb-4661-b079-30a573ee461e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:28.342207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1183/2373] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:28.345301 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e8c1c-3adb-4661-b079-30a573ee461e admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:28.349379 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5f4eb-8b66-4246-a35d-350044dd64a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:28.350001 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5f4eb-8b66-4246-a35d-350044dd64a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:28.350276 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5f4eb-8b66-4246-a35d-350044dd64a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:28.350544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5f4eb-8b66-4246-a35d-350044dd64a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:28.352289 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e4c5f4eb-8b66-4246-a35d-350044dd64a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:28.355439 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f38efb-5027-486f-a6fb-ef7da97cf0a6 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:28.356868 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-24f38efb-5027-486f-a6fb-ef7da97cf0a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:28.424920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f38efb-5027-486f-a6fb-ef7da97cf0a6 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:28.429209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8254e7-d339-478c-a965-bd77675a6919 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:28.429897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8254e7-d339-478c-a965-bd77675a6919 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:28.430238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8254e7-d339-478c-a965-bd77675a6919 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:28.430588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8254e7-d339-478c-a965-bd77675a6919 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:28.432403 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5f4eb-8b66-4246-a35d-350044dd64a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:28.434114 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c5f4eb-8b66-4246-a35d-350044dd64a5 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:28.441233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8254e7-d339-478c-a965-bd77675a6919 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:28.442970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8254e7-d339-478c-a965-bd77675a6919 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:28.444438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1191/2374] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:23:28.452487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1184/2375] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:23:28.890089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5f4eb-8b66-4246-a35d-350044dd64a5 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:28.901280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5f4eb-8b66-4246-a35d-350044dd64a5 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:28.905736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2e466-312b-47f1-a7f9-bb031b2b2e10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:28.906491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2e466-312b-47f1-a7f9-bb031b2b2e10 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:28.906770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2e466-312b-47f1-a7f9-bb031b2b2e10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:28.907044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2e466-312b-47f1-a7f9-bb031b2b2e10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:28.908968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-88a2e466-312b-47f1-a7f9-bb031b2b2e10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:28.926324 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2e466-312b-47f1-a7f9-bb031b2b2e10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:28.928194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2e466-312b-47f1-a7f9-bb031b2b2e10 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:28.940945 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1192/2376] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:30.029801 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c2cc7-6a65-4b59-8552-86d8306e6992 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:30.030297 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c2cc7-6a65-4b59-8552-86d8306e6992 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:30.030384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c2cc7-6a65-4b59-8552-86d8306e6992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:30.051012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-fe4c2cc7-6a65-4b59-8552-86d8306e6992 None None] MFA Rules not processed for user `937131b7fd0048bf97cc2d0b5c3504f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:30.052513 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-fe4c2cc7-6a65-4b59-8552-86d8306e6992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:30.060270 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1185/2377] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:31.922141 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2e466-312b-47f1-a7f9-bb031b2b2e10 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:31.923997 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-88a2e466-312b-47f1-a7f9-bb031b2b2e10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:32.051890 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2e466-312b-47f1-a7f9-bb031b2b2e10 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:32.069402 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b870bba-c48f-4464-be1d-0913bcb39357 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796772cf491047c8b0a609dae9402307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:32.070057 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b870bba-c48f-4464-be1d-0913bcb39357 service placement] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:32.070334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b870bba-c48f-4464-be1d-0913bcb39357 service placement] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:32.070604 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b870bba-c48f-4464-be1d-0913bcb39357 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:32.073396 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-5b870bba-c48f-4464-be1d-0913bcb39357 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:32.097455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b870bba-c48f-4464-be1d-0913bcb39357 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:32.099089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b870bba-c48f-4464-be1d-0913bcb39357 service placement] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:32.117859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1193/2378] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:32.434896 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c2cc7-6a65-4b59-8552-86d8306e6992 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:32.439289 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-fe4c2cc7-6a65-4b59-8552-86d8306e6992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:32.462783 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c2cc7-6a65-4b59-8552-86d8306e6992 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:32.470852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd095e1d-5226-4c19-8caa-b4f1cb3922c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:32.473066 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd095e1d-5226-4c19-8caa-b4f1cb3922c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:32.473680 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd095e1d-5226-4c19-8caa-b4f1cb3922c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:32.474872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd095e1d-5226-4c19-8caa-b4f1cb3922c8 admin admin] PATH_INFO: `/v3/users/76703dc82dc644ef8d32f7e259ddea6f` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:32.479559 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd095e1d-5226-4c19-8caa-b4f1cb3922c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76703dc82dc644ef8d32f7e259ddea6f)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:32.482261 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd095e1d-5226-4c19-8caa-b4f1cb3922c8 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:32.569487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-fd095e1d-5226-4c19-8caa-b4f1cb3922c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76703dc82dc644ef8d32f7e259ddea6f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:32.570310 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-fd095e1d-5226-4c19-8caa-b4f1cb3922c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76703dc82dc644ef8d32f7e259ddea6f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:32.571751 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-fd095e1d-5226-4c19-8caa-b4f1cb3922c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76703dc82dc644ef8d32f7e259ddea6f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:32.575270 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:32.591670 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-fd095e1d-5226-4c19-8caa-b4f1cb3922c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76703dc82dc644ef8d32f7e259ddea6f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:32.603612 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-fd095e1d-5226-4c19-8caa-b4f1cb3922c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76703dc82dc644ef8d32f7e259ddea6f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:32.635756 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1186/2379] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:23:32 2022] DELETE /identity/v3/users/76703dc82dc644ef8d32f7e259ddea6f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:33.299488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b870bba-c48f-4464-be1d-0913bcb39357 service placement] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:33.319130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b870bba-c48f-4464-be1d-0913bcb39357 service placement] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:33.366332 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:33.458140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1411cc16-710c-45e0-a1aa-dac0f8968888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:33.459201 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411cc16-710c-45e0-a1aa-dac0f8968888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:33.459540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411cc16-710c-45e0-a1aa-dac0f8968888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:33.459826 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1411cc16-710c-45e0-a1aa-dac0f8968888 admin admin] PATH_INFO: `/v3/projects/2f003cc2b7104c369e1aa74d6e48a9fc` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:33.461156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411cc16-710c-45e0-a1aa-dac0f8968888 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f003cc2b7104c369e1aa74d6e48a9fc)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:33.463129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1411cc16-710c-45e0-a1aa-dac0f8968888 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:33.562186 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1411cc16-710c-45e0-a1aa-dac0f8968888 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f003cc2b7104c369e1aa74d6e48a9fc', 'request_id': 'req-1411cc16-710c-45e0-a1aa-dac0f8968888'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:33.573765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1411cc16-710c-45e0-a1aa-dac0f8968888 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f003cc2b7104c369e1aa74d6e48a9fc', 'request_id': 'req-1411cc16-710c-45e0-a1aa-dac0f8968888'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:33.586276 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1411cc16-710c-45e0-a1aa-dac0f8968888 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f003cc2b7104c369e1aa74d6e48a9fc', 'request_id': 'req-1411cc16-710c-45e0-a1aa-dac0f8968888'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:33.611409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1411cc16-710c-45e0-a1aa-dac0f8968888 admin admin] The token cache is being invalidate because project 2f003cc2b7104c369e1aa74d6e48a9fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:23:33.612014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1411cc16-710c-45e0-a1aa-dac0f8968888 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:33.612926 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1411cc16-710c-45e0-a1aa-dac0f8968888 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:33.619758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1194/2380] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:23:33 2022] DELETE /identity/v3/projects/2f003cc2b7104c369e1aa74d6e48a9fc => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:33.621337 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f86e5-4cde-4274-9986-392421431b4c None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:33.622416 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f86e5-4cde-4274-9986-392421431b4c None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:33.622872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f86e5-4cde-4274-9986-392421431b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:33.668119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-dc4f86e5-4cde-4274-9986-392421431b4c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:33.669428 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-dc4f86e5-4cde-4274-9986-392421431b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:33.773228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1187/2381] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:33.775078 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc119d5-eff3-4466-a5d1-d074d2065ce2 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:33.775362 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc119d5-eff3-4466-a5d1-d074d2065ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:33.775635 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc119d5-eff3-4466-a5d1-d074d2065ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:33.797310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-2dc119d5-eff3-4466-a5d1-d074d2065ce2 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:33.798701 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-2dc119d5-eff3-4466-a5d1-d074d2065ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:33.808176 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1195/2382] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:33.812316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f86e5-4cde-4274-9986-392421431b4c None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:33.824798 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f86e5-4cde-4274-9986-392421431b4c None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:33.832503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef34f5-40ab-4c36-a9ec-e09f8d8b5a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:33.833166 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef34f5-40ab-4c36-a9ec-e09f8d8b5a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:33.833446 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef34f5-40ab-4c36-a9ec-e09f8d8b5a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:33.833717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef34f5-40ab-4c36-a9ec-e09f8d8b5a91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:33.834415 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef34f5-40ab-4c36-a9ec-e09f8d8b5a91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:23:33.834715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef34f5-40ab-4c36-a9ec-e09f8d8b5a91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:33.836831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef34f5-40ab-4c36-a9ec-e09f8d8b5a91 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:33.873566 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1188/2383] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:23:33.878801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc119d5-eff3-4466-a5d1-d074d2065ce2 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:33.903136 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc119d5-eff3-4466-a5d1-d074d2065ce2 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:33.929907 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a79302-6152-4fda-afc8-f67f53af77d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:33.930539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a79302-6152-4fda-afc8-f67f53af77d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:33.930820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a79302-6152-4fda-afc8-f67f53af77d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:33.931100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a79302-6152-4fda-afc8-f67f53af77d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:33.931779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a79302-6152-4fda-afc8-f67f53af77d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:33.934511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a79302-6152-4fda-afc8-f67f53af77d1 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:33.990777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1196/2384] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:23:33 2022] POST /identity/v3/projects => generated 375 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:33.994845 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef34f5-40ab-4c36-a9ec-e09f8d8b5a91 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:34.013483 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef34f5-40ab-4c36-a9ec-e09f8d8b5a91 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:34.104065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40155755-7f9f-497c-aed5-f7addf3e62a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:34.104900 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40155755-7f9f-497c-aed5-f7addf3e62a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:34.105189 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40155755-7f9f-497c-aed5-f7addf3e62a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:34.105466 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40155755-7f9f-497c-aed5-f7addf3e62a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:34.106147 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40155755-7f9f-497c-aed5-f7addf3e62a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:34.108514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40155755-7f9f-497c-aed5-f7addf3e62a0 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:34.136176 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1189/2385] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:23:33 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:34.141200 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a79302-6152-4fda-afc8-f67f53af77d1 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:34.155155 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a79302-6152-4fda-afc8-f67f53af77d1 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:34.161159 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5e706-6945-4bd0-8d46-10109df23be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:34.161916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5e706-6945-4bd0-8d46-10109df23be7 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:34.162267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5e706-6945-4bd0-8d46-10109df23be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:34.162609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e5e706-6945-4bd0-8d46-10109df23be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:34.163384 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5e706-6945-4bd0-8d46-10109df23be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:34.165123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e5e706-6945-4bd0-8d46-10109df23be7 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:34.181447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1197/2386] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:34.185350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40155755-7f9f-497c-aed5-f7addf3e62a0 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:34.199686 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40155755-7f9f-497c-aed5-f7addf3e62a0 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:34.207016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e342a-99b3-41ca-8d9e-c756b43c65d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:34.208146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e342a-99b3-41ca-8d9e-c756b43c65d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:34.209032 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e342a-99b3-41ca-8d9e-c756b43c65d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:34.209550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e342a-99b3-41ca-8d9e-c756b43c65d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:34.210748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e342a-99b3-41ca-8d9e-c756b43c65d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:34.213038 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29e342a-99b3-41ca-8d9e-c756b43c65d3 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:34.230452 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1190/2387] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:34.235710 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5e706-6945-4bd0-8d46-10109df23be7 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:34.248853 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e5e706-6945-4bd0-8d46-10109df23be7 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:34.255788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8387c2-9215-4927-846f-79d523bb4dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:34.256833 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8387c2-9215-4927-846f-79d523bb4dda admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:34.257323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8387c2-9215-4927-846f-79d523bb4dda admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:34.257701 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8387c2-9215-4927-846f-79d523bb4dda admin admin] PATH_INFO: `/v3/projects/0846ad04fd624ae6b5a4878f2ffb021b/users/5a9e1d1f299a46e69bf7fefbfa49d2e1/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:34.281636 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8387c2-9215-4927-846f-79d523bb4dda admin admin] RBAC: Authorizing `identity:create_grant(project_id=0846ad04fd624ae6b5a4878f2ffb021b, user_id=5a9e1d1f299a46e69bf7fefbfa49d2e1, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:34.284711 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8387c2-9215-4927-846f-79d523bb4dda admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:34.306859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1198/2388] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:34 2022] PUT /identity/v3/projects/0846ad04fd624ae6b5a4878f2ffb021b/users/5a9e1d1f299a46e69bf7fefbfa49d2e1/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:34.317949 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06241ba5-c50f-4b41-ae84-3c5c4c3c36b6 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:34.318387 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06241ba5-c50f-4b41-ae84-3c5c4c3c36b6 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:34.318814 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06241ba5-c50f-4b41-ae84-3c5c4c3c36b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:34.411799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-06241ba5-c50f-4b41-ae84-3c5c4c3c36b6 None None] MFA Rules not processed for user `5a9e1d1f299a46e69bf7fefbfa49d2e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:34.413494 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-06241ba5-c50f-4b41-ae84-3c5c4c3c36b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:34.509606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:34.610449 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1191/2389] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:34.620099 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8387c2-9215-4927-846f-79d523bb4dda admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:34.622052 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-ef8387c2-9215-4927-846f-79d523bb4dda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:34.720263 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:34.769946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8387c2-9215-4927-846f-79d523bb4dda admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:34.783850 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ff54a-5c3e-4786-8d3c-ae93778a45d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:34.784840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ff54a-5c3e-4786-8d3c-ae93778a45d2 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:34.785348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ff54a-5c3e-4786-8d3c-ae93778a45d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:34.785825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ff54a-5c3e-4786-8d3c-ae93778a45d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:34.817269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ff54a-5c3e-4786-8d3c-ae93778a45d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:34.819408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ff54a-5c3e-4786-8d3c-ae93778a45d2 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:34.851292 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1199/2390] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:35.153366 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06241ba5-c50f-4b41-ae84-3c5c4c3c36b6 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:35.242232 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06241ba5-c50f-4b41-ae84-3c5c4c3c36b6 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:35.248219 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74f15d-6218-4dfe-9c0e-c4de2858a6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:35.248936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74f15d-6218-4dfe-9c0e-c4de2858a6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:35.249218 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74f15d-6218-4dfe-9c0e-c4de2858a6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:35.249534 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74f15d-6218-4dfe-9c0e-c4de2858a6ab admin admin] PATH_INFO: `/v3/users/5a9e1d1f299a46e69bf7fefbfa49d2e1` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:35.250883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74f15d-6218-4dfe-9c0e-c4de2858a6ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a9e1d1f299a46e69bf7fefbfa49d2e1)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:35.252877 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74f15d-6218-4dfe-9c0e-c4de2858a6ab admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:35.327331 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-7c74f15d-6218-4dfe-9c0e-c4de2858a6ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a9e1d1f299a46e69bf7fefbfa49d2e1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:35.328881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-7c74f15d-6218-4dfe-9c0e-c4de2858a6ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a9e1d1f299a46e69bf7fefbfa49d2e1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:35.329890 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-7c74f15d-6218-4dfe-9c0e-c4de2858a6ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a9e1d1f299a46e69bf7fefbfa49d2e1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:35.361341 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-7c74f15d-6218-4dfe-9c0e-c4de2858a6ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a9e1d1f299a46e69bf7fefbfa49d2e1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:35.369855 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-7c74f15d-6218-4dfe-9c0e-c4de2858a6ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a9e1d1f299a46e69bf7fefbfa49d2e1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:35.399311 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1192/2391] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:23:35 2022] DELETE /identity/v3/users/5a9e1d1f299a46e69bf7fefbfa49d2e1 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:35.408905 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9acb4-db19-44d9-ab3e-90cd47d22a95 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:35.408905 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9acb4-db19-44d9-ab3e-90cd47d22a95 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:35.409465 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9acb4-db19-44d9-ab3e-90cd47d22a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:35.430625 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-18a9acb4-db19-44d9-ab3e-90cd47d22a95 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:35.432206 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-18a9acb4-db19-44d9-ab3e-90cd47d22a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:35.522995 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74f15d-6218-4dfe-9c0e-c4de2858a6ab admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:35.525198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7c74f15d-6218-4dfe-9c0e-c4de2858a6ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:35.527814 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1200/2392] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:35.548820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74f15d-6218-4dfe-9c0e-c4de2858a6ab admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:35.554669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9acb4-db19-44d9-ab3e-90cd47d22a95 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:35.556427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-18a9acb4-db19-44d9-ab3e-90cd47d22a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:35.563147 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3c0a6-ebb4-48bf-9f18-79dee9e8c7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:35.564014 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3c0a6-ebb4-48bf-9f18-79dee9e8c7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:35.564142 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3c0a6-ebb4-48bf-9f18-79dee9e8c7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:35.564364 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d3c0a6-ebb4-48bf-9f18-79dee9e8c7c6 admin admin] PATH_INFO: `/v3/users/26e73db77d93496883b66572d6bf35f2` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:35.565609 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3c0a6-ebb4-48bf-9f18-79dee9e8c7c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26e73db77d93496883b66572d6bf35f2)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:35.568241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d3c0a6-ebb4-48bf-9f18-79dee9e8c7c6 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:35.643688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9acb4-db19-44d9-ab3e-90cd47d22a95 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:35.651689 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6e59b-46db-4885-b197-2207babbe088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:35.652315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6e59b-46db-4885-b197-2207babbe088 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:35.652966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6e59b-46db-4885-b197-2207babbe088 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:35.653436 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b6e59b-46db-4885-b197-2207babbe088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:35.668497 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:35.668837 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6e59b-46db-4885-b197-2207babbe088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:35.670541 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b6e59b-46db-4885-b197-2207babbe088 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:35.681540 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1201/2393] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:23:35.685860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-09d3c0a6-ebb4-48bf-9f18-79dee9e8c7c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26e73db77d93496883b66572d6bf35f2'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:35.686823 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-09d3c0a6-ebb4-48bf-9f18-79dee9e8c7c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26e73db77d93496883b66572d6bf35f2'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:35.688146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-09d3c0a6-ebb4-48bf-9f18-79dee9e8c7c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26e73db77d93496883b66572d6bf35f2'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:35.721563 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-09d3c0a6-ebb4-48bf-9f18-79dee9e8c7c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26e73db77d93496883b66572d6bf35f2'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:35.736266 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-09d3c0a6-ebb4-48bf-9f18-79dee9e8c7c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26e73db77d93496883b66572d6bf35f2'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:35.763554 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1193/2394] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:23:35 2022] DELETE /identity/v3/users/26e73db77d93496883b66572d6bf35f2 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:36.053304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6e59b-46db-4885-b197-2207babbe088 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:36.059630 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3c0a6-ebb4-48bf-9f18-79dee9e8c7c6 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:36.061498 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-09d3c0a6-ebb4-48bf-9f18-79dee9e8c7c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:36.070518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b6e59b-46db-4885-b197-2207babbe088 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:36.168096 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042bc316-45a5-4bb4-a60f-e16c9429fa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:36.169028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d3c0a6-ebb4-48bf-9f18-79dee9e8c7c6 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:36.169510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bc316-45a5-4bb4-a60f-e16c9429fa62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:36.169782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bc316-45a5-4bb4-a60f-e16c9429fa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:36.169991 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042bc316-45a5-4bb4-a60f-e16c9429fa62 admin admin] PATH_INFO: `/v3/projects/1c32d82cce6541718e46908eaf9b98e9` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:36.171742 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042bc316-45a5-4bb4-a60f-e16c9429fa62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c32d82cce6541718e46908eaf9b98e9)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:36.173776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042bc316-45a5-4bb4-a60f-e16c9429fa62 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:36.179976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc2881-596d-4487-b2d5-f1a9606041c4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796772cf491047c8b0a609dae9402307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:36.181036 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc2881-596d-4487-b2d5-f1a9606041c4 service placement] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:36.181480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc2881-596d-4487-b2d5-f1a9606041c4 service placement] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:36.181979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc2881-596d-4487-b2d5-f1a9606041c4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:36.184560 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e1cc2881-596d-4487-b2d5-f1a9606041c4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:36.206122 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:36.220533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc2881-596d-4487-b2d5-f1a9606041c4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:36.223404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cc2881-596d-4487-b2d5-f1a9606041c4 service placement] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:36.248783 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1194/2395] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:23:36.256048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-042bc316-45a5-4bb4-a60f-e16c9429fa62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c32d82cce6541718e46908eaf9b98e9', 'request_id': 'req-042bc316-45a5-4bb4-a60f-e16c9429fa62'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:36.271634 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-042bc316-45a5-4bb4-a60f-e16c9429fa62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c32d82cce6541718e46908eaf9b98e9', 'request_id': 'req-042bc316-45a5-4bb4-a60f-e16c9429fa62'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:36.282968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-042bc316-45a5-4bb4-a60f-e16c9429fa62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c32d82cce6541718e46908eaf9b98e9', 'request_id': 'req-042bc316-45a5-4bb4-a60f-e16c9429fa62'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:36.318722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-042bc316-45a5-4bb4-a60f-e16c9429fa62 admin admin] The token cache is being invalidate because project 1c32d82cce6541718e46908eaf9b98e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:23:36.321059 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-042bc316-45a5-4bb4-a60f-e16c9429fa62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:36.322265 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-042bc316-45a5-4bb4-a60f-e16c9429fa62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:36.329093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1202/2396] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:23:36 2022] DELETE /identity/v3/projects/1c32d82cce6541718e46908eaf9b98e9 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:36.334034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64fcd9-0e5e-419f-8f9f-7d430394dde6 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:36.334510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64fcd9-0e5e-419f-8f9f-7d430394dde6 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:36.335435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64fcd9-0e5e-419f-8f9f-7d430394dde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:36.339854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042bc316-45a5-4bb4-a60f-e16c9429fa62 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:36.345332 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-042bc316-45a5-4bb4-a60f-e16c9429fa62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:36.370872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-8f64fcd9-0e5e-419f-8f9f-7d430394dde6 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:36.372079 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-8f64fcd9-0e5e-419f-8f9f-7d430394dde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:36.480666 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1195/2397] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:36.482521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc6fed-bb88-4355-923d-6c94b921230b None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:36.483006 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042bc316-45a5-4bb4-a60f-e16c9429fa62 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:36.485025 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc6fed-bb88-4355-923d-6c94b921230b None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:36.485385 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc6fed-bb88-4355-923d-6c94b921230b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:36.487773 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e03eb-431c-4589-a831-d44faaaec5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:36.488417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e03eb-431c-4589-a831-d44faaaec5f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:36.488735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e03eb-431c-4589-a831-d44faaaec5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:36.489010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e03eb-431c-4589-a831-d44faaaec5f5 admin admin] PATH_INFO: `/v3/projects/0846ad04fd624ae6b5a4878f2ffb021b` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:36.490178 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e03eb-431c-4589-a831-d44faaaec5f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0846ad04fd624ae6b5a4878f2ffb021b)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:36.492074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e03eb-431c-4589-a831-d44faaaec5f5 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:36.510544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-6dfc6fed-bb88-4355-923d-6c94b921230b None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:36.512414 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-6dfc6fed-bb88-4355-923d-6c94b921230b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:36.527290 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1196/2398] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:36.528324 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc6fed-bb88-4355-923d-6c94b921230b None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:36.539545 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc6fed-bb88-4355-923d-6c94b921230b None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:36.545222 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db657af-273b-4d8a-b148-6823ea0393a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:36.545936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db657af-273b-4d8a-b148-6823ea0393a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:36.547345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db657af-273b-4d8a-b148-6823ea0393a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:36.547345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db657af-273b-4d8a-b148-6823ea0393a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:36.548067 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db657af-273b-4d8a-b148-6823ea0393a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:23:36.548325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db657af-273b-4d8a-b148-6823ea0393a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:36.551390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db657af-273b-4d8a-b148-6823ea0393a7 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:36.571191 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1197/2399] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:23:36.572884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b81e03eb-431c-4589-a831-d44faaaec5f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0846ad04fd624ae6b5a4878f2ffb021b', 'request_id': 'req-b81e03eb-431c-4589-a831-d44faaaec5f5'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:36.575741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db657af-273b-4d8a-b148-6823ea0393a7 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:36.587533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b81e03eb-431c-4589-a831-d44faaaec5f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0846ad04fd624ae6b5a4878f2ffb021b', 'request_id': 'req-b81e03eb-431c-4589-a831-d44faaaec5f5'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:36.589092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db657af-273b-4d8a-b148-6823ea0393a7 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:36.597458 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b81e03eb-431c-4589-a831-d44faaaec5f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0846ad04fd624ae6b5a4878f2ffb021b', 'request_id': 'req-b81e03eb-431c-4589-a831-d44faaaec5f5'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:36.613659 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b81e03eb-431c-4589-a831-d44faaaec5f5 admin admin] The token cache is being invalidate because project 0846ad04fd624ae6b5a4878f2ffb021b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:23:36.615214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b81e03eb-431c-4589-a831-d44faaaec5f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:36.615214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-b81e03eb-431c-4589-a831-d44faaaec5f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:36.616588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1203/2400] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:23:36 2022] DELETE /identity/v3/projects/0846ad04fd624ae6b5a4878f2ffb021b => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:36.619410 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:36.621512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd653da8-9775-43c7-b121-0151589ad1df None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:36.621985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd653da8-9775-43c7-b121-0151589ad1df None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:36.622433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd653da8-9775-43c7-b121-0151589ad1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:36.657511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-bd653da8-9775-43c7-b121-0151589ad1df None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:36.661453 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-bd653da8-9775-43c7-b121-0151589ad1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:36.680392 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c9f0a-0f9b-4eb1-a2aa-08525330e207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:36.682090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c9f0a-0f9b-4eb1-a2aa-08525330e207 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:36.682706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c9f0a-0f9b-4eb1-a2aa-08525330e207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:36.682706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c9f0a-0f9b-4eb1-a2aa-08525330e207 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:36.683599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c9f0a-0f9b-4eb1-a2aa-08525330e207 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:36.685615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c9f0a-0f9b-4eb1-a2aa-08525330e207 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:36.714992 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:36.732232 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1198/2401] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:23:36 2022] POST /identity/v3/projects => generated 403 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:36.736894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c9f0a-0f9b-4eb1-a2aa-08525330e207 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:36.739652 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9c1c9f0a-0f9b-4eb1-a2aa-08525330e207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:36.764982 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1204/2402] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:23:36.766473 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9778e-d540-4043-86ad-ae71741e22ac None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:36.766770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9778e-d540-4043-86ad-ae71741e22ac None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:36.767052 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e9778e-d540-4043-86ad-ae71741e22ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:36.789672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-80e9778e-d540-4043-86ad-ae71741e22ac None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:36.791807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-80e9778e-d540-4043-86ad-ae71741e22ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:36.835635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c9f0a-0f9b-4eb1-a2aa-08525330e207 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:36.841126 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9233ef-e16b-4973-939e-4a9904295e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:36.841750 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9233ef-e16b-4973-939e-4a9904295e1b admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:36.842028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9233ef-e16b-4973-939e-4a9904295e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:36.842301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9233ef-e16b-4973-939e-4a9904295e1b admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:36.842966 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9233ef-e16b-4973-939e-4a9904295e1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:36.845523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9233ef-e16b-4973-939e-4a9904295e1b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:36.872144 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1205/2403] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:36.877229 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9778e-d540-4043-86ad-ae71741e22ac None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:36.885799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1199/2404] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:23:36 2022] POST /identity/v3/users => generated 302 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:23:36.888521 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e9778e-d540-4043-86ad-ae71741e22ac None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:36.889864 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9233ef-e16b-4973-939e-4a9904295e1b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:36.894566 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a15347-aed4-4220-b78b-cba5a01b5e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:36.895502 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a15347-aed4-4220-b78b-cba5a01b5e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:36.896019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a15347-aed4-4220-b78b-cba5a01b5e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:36.896932 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a15347-aed4-4220-b78b-cba5a01b5e3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:36.897796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a15347-aed4-4220-b78b-cba5a01b5e3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:23:36.898195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a15347-aed4-4220-b78b-cba5a01b5e3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:36.899885 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a15347-aed4-4220-b78b-cba5a01b5e3d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:36.900044 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9233ef-e16b-4973-939e-4a9904295e1b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:36.906185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b329796-3e39-4a24-b457-69789fdd34e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:36.906843 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b329796-3e39-4a24-b457-69789fdd34e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:36.907128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b329796-3e39-4a24-b457-69789fdd34e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:36.907402 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b329796-3e39-4a24-b457-69789fdd34e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:36.908650 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b329796-3e39-4a24-b457-69789fdd34e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:36.910269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b329796-3e39-4a24-b457-69789fdd34e9 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:36.913673 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1206/2405] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:23:36.917982 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a15347-aed4-4220-b78b-cba5a01b5e3d admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:36.923546 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1200/2406] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:23:36.928056 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b329796-3e39-4a24-b457-69789fdd34e9 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:36.930583 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a15347-aed4-4220-b78b-cba5a01b5e3d admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:36.936690 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b6428-388b-4e4f-864f-9a2cc79433c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:36.938124 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b6428-388b-4e4f-864f-9a2cc79433c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:36.938507 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b6428-388b-4e4f-864f-9a2cc79433c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:36.939853 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b6428-388b-4e4f-864f-9a2cc79433c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:36.939980 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b329796-3e39-4a24-b457-69789fdd34e9 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:36.940083 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b6428-388b-4e4f-864f-9a2cc79433c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:36.941814 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b6428-388b-4e4f-864f-9a2cc79433c2 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:36.944036 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89b271-eb47-43a0-ab3e-4ce527ffbb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:36.944698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89b271-eb47-43a0-ab3e-4ce527ffbb1d admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:36.944988 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89b271-eb47-43a0-ab3e-4ce527ffbb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:36.945265 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89b271-eb47-43a0-ab3e-4ce527ffbb1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:36.945961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89b271-eb47-43a0-ab3e-4ce527ffbb1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:36.947830 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89b271-eb47-43a0-ab3e-4ce527ffbb1d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:36.966487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1201/2407] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:23:36.973156 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89b271-eb47-43a0-ab3e-4ce527ffbb1d admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:36.979828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1207/2408] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:23:36 2022] POST /identity/v3/projects => generated 405 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:36.985033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b6428-388b-4e4f-864f-9a2cc79433c2 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:36.994025 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89b271-eb47-43a0-ab3e-4ce527ffbb1d admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:37.056177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b6428-388b-4e4f-864f-9a2cc79433c2 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:37.061270 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a596b5-1cc2-4ad2-8915-5f42f5f38cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:37.062095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a596b5-1cc2-4ad2-8915-5f42f5f38cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:37.062479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a596b5-1cc2-4ad2-8915-5f42f5f38cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:37.063003 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a596b5-1cc2-4ad2-8915-5f42f5f38cf1 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:37.063791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a596b5-1cc2-4ad2-8915-5f42f5f38cf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:37.065964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a596b5-1cc2-4ad2-8915-5f42f5f38cf1 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:37.097989 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1208/2409] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:23:36 2022] POST /identity/v3/users => generated 303 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:37.099246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1b933-b6c6-4120-99bf-56099c5db4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:37.100361 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1b933-b6c6-4120-99bf-56099c5db4d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:37.100916 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1b933-b6c6-4120-99bf-56099c5db4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:37.101425 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1b933-b6c6-4120-99bf-56099c5db4d7 admin admin] PATH_INFO: `/v3/projects/9db150ab0ee44ac6bb14ff2eab6fd197/users/d72df516104e42f4b154857cf8a33641/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:37.107861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a596b5-1cc2-4ad2-8915-5f42f5f38cf1 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:37.121283 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a596b5-1cc2-4ad2-8915-5f42f5f38cf1 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:37.123375 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1b933-b6c6-4120-99bf-56099c5db4d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9db150ab0ee44ac6bb14ff2eab6fd197, user_id=d72df516104e42f4b154857cf8a33641, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:37.128562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1b933-b6c6-4120-99bf-56099c5db4d7 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:37.129455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48f139-cff4-4ce4-8446-4789bdf90e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:37.130548 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48f139-cff4-4ce4-8446-4789bdf90e6d admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:37.131098 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48f139-cff4-4ce4-8446-4789bdf90e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:37.131631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48f139-cff4-4ce4-8446-4789bdf90e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:37.132946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48f139-cff4-4ce4-8446-4789bdf90e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:37.135951 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48f139-cff4-4ce4-8446-4789bdf90e6d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:37.150312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:37.156488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1209/2410] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:23:37.156706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1202/2411] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:36 2022] PUT /identity/v3/projects/9db150ab0ee44ac6bb14ff2eab6fd197/users/d72df516104e42f4b154857cf8a33641/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:37.157901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48f139-cff4-4ce4-8446-4789bdf90e6d admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:37.167556 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf0e13-848e-44c3-854b-31da5b1e259b None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:37.167967 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf0e13-848e-44c3-854b-31da5b1e259b None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:37.168411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf0e13-848e-44c3-854b-31da5b1e259b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:37.168652 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48f139-cff4-4ce4-8446-4789bdf90e6d admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:37.188788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:37.200979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-7faf0e13-848e-44c3-854b-31da5b1e259b None None] MFA Rules not processed for user `d72df516104e42f4b154857cf8a33641`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:37.202407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7faf0e13-848e-44c3-854b-31da5b1e259b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:37.236721 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63ed9b-838a-42fa-a7f2-a2d941caaa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:37.237767 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63ed9b-838a-42fa-a7f2-a2d941caaa80 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:37.238052 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63ed9b-838a-42fa-a7f2-a2d941caaa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:37.238331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed63ed9b-838a-42fa-a7f2-a2d941caaa80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:37.239029 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63ed9b-838a-42fa-a7f2-a2d941caaa80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:37.240796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed63ed9b-838a-42fa-a7f2-a2d941caaa80 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:37.255349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1210/2412] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:37.256639 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63ed9b-838a-42fa-a7f2-a2d941caaa80 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:37.267596 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed63ed9b-838a-42fa-a7f2-a2d941caaa80 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:37.272665 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc4550-eecd-42cc-b9b7-94b382698438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:37.273049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc4550-eecd-42cc-b9b7-94b382698438 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:37.273410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc4550-eecd-42cc-b9b7-94b382698438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:37.273723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc4550-eecd-42cc-b9b7-94b382698438 admin admin] PATH_INFO: `/v3/projects/6e0e0182d3494f08a3b84b78740ae151/users/29b3c639d0514b63b0f5976560e7663e/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:37.279255 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1203/2413] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:37 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:23:37.284136 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf0e13-848e-44c3-854b-31da5b1e259b None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:37.295412 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc4550-eecd-42cc-b9b7-94b382698438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e0e0182d3494f08a3b84b78740ae151, user_id=29b3c639d0514b63b0f5976560e7663e, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:37.295948 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf0e13-848e-44c3-854b-31da5b1e259b None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:37.298896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bc4550-eecd-42cc-b9b7-94b382698438 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:37.301329 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f861d58-fc18-479c-88c1-2fc346611b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:37.301952 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f861d58-fc18-479c-88c1-2fc346611b44 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:37.302238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f861d58-fc18-479c-88c1-2fc346611b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:37.302517 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f861d58-fc18-479c-88c1-2fc346611b44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:37.303268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f861d58-fc18-479c-88c1-2fc346611b44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:37.305511 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f861d58-fc18-479c-88c1-2fc346611b44 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:37.319641 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1211/2414] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:37 2022] PUT /identity/v3/projects/6e0e0182d3494f08a3b84b78740ae151/users/29b3c639d0514b63b0f5976560e7663e/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:23:37.321673 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e5ade-5330-49bc-94c6-7b4e9e078ab7 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:37.322058 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e5ade-5330-49bc-94c6-7b4e9e078ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:37.322455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e5ade-5330-49bc-94c6-7b4e9e078ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:37.342162 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1204/2415] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:23:37 2022] POST /identity/v3/projects => generated 403 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:23:37.343477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-b31e5ade-5330-49bc-94c6-7b4e9e078ab7 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:37.344872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f861d58-fc18-479c-88c1-2fc346611b44 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:37.345488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b31e5ade-5330-49bc-94c6-7b4e9e078ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:37.418896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1212/2416] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:23:37.425252 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f861d58-fc18-479c-88c1-2fc346611b44 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:37.430709 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72044205-2350-4841-a87e-8511bad13a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:37.431355 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72044205-2350-4841-a87e-8511bad13a6c admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:37.431661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72044205-2350-4841-a87e-8511bad13a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:37.431919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72044205-2350-4841-a87e-8511bad13a6c admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:37.432762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72044205-2350-4841-a87e-8511bad13a6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:37.434849 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72044205-2350-4841-a87e-8511bad13a6c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:37.435306 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e5ade-5330-49bc-94c6-7b4e9e078ab7 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:37.438092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b31e5ade-5330-49bc-94c6-7b4e9e078ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:37.469219 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1205/2417] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:23:37 2022] POST /identity/v3/users => generated 308 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:37.473179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72044205-2350-4841-a87e-8511bad13a6c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:37.487960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72044205-2350-4841-a87e-8511bad13a6c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:37.496069 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fd139-661c-40b4-b30f-e8748c7b7d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:37.496829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fd139-661c-40b4-b30f-e8748c7b7d62 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:37.497182 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fd139-661c-40b4-b30f-e8748c7b7d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:37.497644 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fd139-661c-40b4-b30f-e8748c7b7d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:37.498420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117fd139-661c-40b4-b30f-e8748c7b7d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:37.501810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117fd139-661c-40b4-b30f-e8748c7b7d62 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:37.521016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1206/2418] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:23:37.522420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fd139-661c-40b4-b30f-e8748c7b7d62 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:37.536060 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e5ade-5330-49bc-94c6-7b4e9e078ab7 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:37.542792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d875839-4644-487f-bb3c-7a6762d2dfb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:37.543507 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d875839-4644-487f-bb3c-7a6762d2dfb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:37.543883 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d875839-4644-487f-bb3c-7a6762d2dfb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:37.544219 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d875839-4644-487f-bb3c-7a6762d2dfb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:37.555285 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fd139-661c-40b4-b30f-e8748c7b7d62 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:37.558195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d875839-4644-487f-bb3c-7a6762d2dfb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:37.560405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d875839-4644-487f-bb3c-7a6762d2dfb2 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:37.566263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feea0eb-3fe2-401f-bc51-1219825a0ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:37.567255 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feea0eb-3fe2-401f-bc51-1219825a0ca4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:37.567600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feea0eb-3fe2-401f-bc51-1219825a0ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:37.567975 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feea0eb-3fe2-401f-bc51-1219825a0ca4 admin admin] PATH_INFO: `/v3/projects/b20c8651d8394dea9414bb617516b1fd/users/a4089c3ebc6f4c3994480adcda065c19/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:37.575954 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1213/2419] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:37.587244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feea0eb-3fe2-401f-bc51-1219825a0ca4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b20c8651d8394dea9414bb617516b1fd, user_id=a4089c3ebc6f4c3994480adcda065c19, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:37.590197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feea0eb-3fe2-401f-bc51-1219825a0ca4 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:37.612728 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1207/2420] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:37 2022] PUT /identity/v3/projects/b20c8651d8394dea9414bb617516b1fd/users/a4089c3ebc6f4c3994480adcda065c19/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:37.616055 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d875839-4644-487f-bb3c-7a6762d2dfb2 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:37.629669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d875839-4644-487f-bb3c-7a6762d2dfb2 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:37.677342 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:37.707985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ee400-f9f6-45d5-9bc2-bf1927bcc971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:37.708879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ee400-f9f6-45d5-9bc2-bf1927bcc971 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:37.709166 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ee400-f9f6-45d5-9bc2-bf1927bcc971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:37.709450 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ee400-f9f6-45d5-9bc2-bf1927bcc971 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:37.710195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ee400-f9f6-45d5-9bc2-bf1927bcc971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:37.711852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78ee400-f9f6-45d5-9bc2-bf1927bcc971 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:37.724560 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1214/2421] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:37 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:37.729394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feea0eb-3fe2-401f-bc51-1219825a0ca4 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:37.741954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feea0eb-3fe2-401f-bc51-1219825a0ca4 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:37.747062 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921df6f-4bcc-42d9-aec7-54209d77e9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:37.747862 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921df6f-4bcc-42d9-aec7-54209d77e9dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:37.748218 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921df6f-4bcc-42d9-aec7-54209d77e9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:37.748570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a921df6f-4bcc-42d9-aec7-54209d77e9dd admin admin] PATH_INFO: `/v3/projects/b20c8651d8394dea9414bb617516b1fd/users/a4089c3ebc6f4c3994480adcda065c19/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:37.750921 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921df6f-4bcc-42d9-aec7-54209d77e9dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b20c8651d8394dea9414bb617516b1fd, user_id=a4089c3ebc6f4c3994480adcda065c19, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:37.753887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a921df6f-4bcc-42d9-aec7-54209d77e9dd admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:37.766983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1208/2422] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:37 2022] PUT /identity/v3/projects/b20c8651d8394dea9414bb617516b1fd/users/a4089c3ebc6f4c3994480adcda065c19/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:37.777597 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e64dc-70b4-43af-a23e-02f52eb7fd4e None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:37.777925 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e64dc-70b4-43af-a23e-02f52eb7fd4e None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:37.778151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e64dc-70b4-43af-a23e-02f52eb7fd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:37.811143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-cf3e64dc-70b4-43af-a23e-02f52eb7fd4e None None] MFA Rules not processed for user `a4089c3ebc6f4c3994480adcda065c19`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:37.814408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-cf3e64dc-70b4-43af-a23e-02f52eb7fd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:37.917371 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1215/2423] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:37 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:37.926372 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921df6f-4bcc-42d9-aec7-54209d77e9dd admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:37.928279 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a921df6f-4bcc-42d9-aec7-54209d77e9dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:37.940078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:38.006874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a921df6f-4bcc-42d9-aec7-54209d77e9dd admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:38.012920 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4b53e-f3a6-434b-870c-e4c07da947b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:38.013528 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4b53e-f3a6-434b-870c-e4c07da947b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:38.014256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4b53e-f3a6-434b-870c-e4c07da947b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:38.014256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4b53e-f3a6-434b-870c-e4c07da947b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:38.029672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4b53e-f3a6-434b-870c-e4c07da947b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:38.032522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4b53e-f3a6-434b-870c-e4c07da947b3 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:38.056496 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1209/2424] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:37 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:38.217325 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e64dc-70b4-43af-a23e-02f52eb7fd4e None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:38.304279 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e64dc-70b4-43af-a23e-02f52eb7fd4e None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:38.309458 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d233a2c-bd03-4d1f-9b0a-f056bd9cafda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:38.310101 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d233a2c-bd03-4d1f-9b0a-f056bd9cafda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:38.310376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d233a2c-bd03-4d1f-9b0a-f056bd9cafda admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:38.310808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d233a2c-bd03-4d1f-9b0a-f056bd9cafda admin admin] PATH_INFO: `/v3/users/d72df516104e42f4b154857cf8a33641` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:38.312015 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d233a2c-bd03-4d1f-9b0a-f056bd9cafda admin admin] RBAC: Authorizing `identity:delete_user(user_id=d72df516104e42f4b154857cf8a33641)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:38.314025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d233a2c-bd03-4d1f-9b0a-f056bd9cafda admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:38.372286 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-7d233a2c-bd03-4d1f-9b0a-f056bd9cafda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd72df516104e42f4b154857cf8a33641'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:38.382626 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-7d233a2c-bd03-4d1f-9b0a-f056bd9cafda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd72df516104e42f4b154857cf8a33641'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:38.384174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-7d233a2c-bd03-4d1f-9b0a-f056bd9cafda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd72df516104e42f4b154857cf8a33641'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:38.386877 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:38.400501 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-7d233a2c-bd03-4d1f-9b0a-f056bd9cafda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd72df516104e42f4b154857cf8a33641'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:38.401503 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-7d233a2c-bd03-4d1f-9b0a-f056bd9cafda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd72df516104e42f4b154857cf8a33641'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:38.440135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1216/2425] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:23:38 2022] DELETE /identity/v3/users/d72df516104e42f4b154857cf8a33641 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:38.444504 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5d159-b756-4815-87f6-e1d5f40835dc None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:38.445568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5d159-b756-4815-87f6-e1d5f40835dc None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:38.446232 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f5d159-b756-4815-87f6-e1d5f40835dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:38.477443 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-42f5d159-b756-4815-87f6-e1d5f40835dc None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:38.479886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-42f5d159-b756-4815-87f6-e1d5f40835dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:38.590319 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1210/2426] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:23:38.610836 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d233a2c-bd03-4d1f-9b0a-f056bd9cafda admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:38.613600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7d233a2c-bd03-4d1f-9b0a-f056bd9cafda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:38.716843 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d233a2c-bd03-4d1f-9b0a-f056bd9cafda admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:38.723135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48f69b-15fc-4288-b731-898ff133f3d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:38.724551 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48f69b-15fc-4288-b731-898ff133f3d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:38.725471 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48f69b-15fc-4288-b731-898ff133f3d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:38.725850 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48f69b-15fc-4288-b731-898ff133f3d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:38.741948 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48f69b-15fc-4288-b731-898ff133f3d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:38.745660 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48f69b-15fc-4288-b731-898ff133f3d2 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:38.760742 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1217/2427] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:39.123426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dde14-6db5-4248-a484-ea0c55a5834b None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:39.125698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dde14-6db5-4248-a484-ea0c55a5834b None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:39.126649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dde14-6db5-4248-a484-ea0c55a5834b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:39.162105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-814dde14-6db5-4248-a484-ea0c55a5834b None None] MFA Rules not processed for user `29b3c639d0514b63b0f5976560e7663e`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:39.163397 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-814dde14-6db5-4248-a484-ea0c55a5834b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:39.244021 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1211/2428] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:39 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:39.248407 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48f69b-15fc-4288-b731-898ff133f3d2 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:39.249834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7d48f69b-15fc-4288-b731-898ff133f3d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:39.263609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48f69b-15fc-4288-b731-898ff133f3d2 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:39.269820 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881bd8c-6ff6-4e8b-b390-3d8bfb25df1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:39.270812 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881bd8c-6ff6-4e8b-b390-3d8bfb25df1a admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:39.271098 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881bd8c-6ff6-4e8b-b390-3d8bfb25df1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:39.271376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881bd8c-6ff6-4e8b-b390-3d8bfb25df1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:39.272073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881bd8c-6ff6-4e8b-b390-3d8bfb25df1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:39.274028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1881bd8c-6ff6-4e8b-b390-3d8bfb25df1a admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:39.310056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1218/2429] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:23:39 2022] POST /identity/v3/projects => generated 403 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:39.311258 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dde14-6db5-4248-a484-ea0c55a5834b None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:39.325704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dde14-6db5-4248-a484-ea0c55a5834b None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:39.374392 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:39.393455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbeefc-eb8d-4cf9-90fd-68493240eefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:39.394135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbeefc-eb8d-4cf9-90fd-68493240eefc admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:39.394408 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbeefc-eb8d-4cf9-90fd-68493240eefc admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:39.394844 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbeefc-eb8d-4cf9-90fd-68493240eefc admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:39.395599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbeefc-eb8d-4cf9-90fd-68493240eefc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:39.397630 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbeefc-eb8d-4cf9-90fd-68493240eefc admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:39.427220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1212/2430] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:23:39 2022] POST /identity/v3/users => generated 308 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:23:39.431337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881bd8c-6ff6-4e8b-b390-3d8bfb25df1a admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:39.445341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881bd8c-6ff6-4e8b-b390-3d8bfb25df1a admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:39.454268 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e11b4-bc25-47f1-b347-c16024b65abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:39.455361 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e11b4-bc25-47f1-b347-c16024b65abb admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:39.455718 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e11b4-bc25-47f1-b347-c16024b65abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:39.456070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1e11b4-bc25-47f1-b347-c16024b65abb admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:39.456978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e11b4-bc25-47f1-b347-c16024b65abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:39.458907 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1e11b4-bc25-47f1-b347-c16024b65abb admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:39.474523 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1219/2431] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:39.479219 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbeefc-eb8d-4cf9-90fd-68493240eefc admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:39.491980 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbeefc-eb8d-4cf9-90fd-68493240eefc admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:39.496978 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafa2d6-6432-4b8d-b83a-4e69418e04c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:39.497665 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa2d6-6432-4b8d-b83a-4e69418e04c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:39.497942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa2d6-6432-4b8d-b83a-4e69418e04c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:39.498215 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafa2d6-6432-4b8d-b83a-4e69418e04c4 admin admin] PATH_INFO: `/v3/projects/46e2257762a34e65b114aa64f3253ad4/users/998220a0ea9944bdbf9269e514330d34/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:39.515396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafa2d6-6432-4b8d-b83a-4e69418e04c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e2257762a34e65b114aa64f3253ad4, user_id=998220a0ea9944bdbf9269e514330d34, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:39.518415 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafa2d6-6432-4b8d-b83a-4e69418e04c4 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:39.544349 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1213/2432] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:39 2022] PUT /identity/v3/projects/46e2257762a34e65b114aa64f3253ad4/users/998220a0ea9944bdbf9269e514330d34/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:39.549074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e11b4-bc25-47f1-b347-c16024b65abb admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:39.557114 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafa2d6-6432-4b8d-b83a-4e69418e04c4 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:39.559078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-aaafa2d6-6432-4b8d-b83a-4e69418e04c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:39.560523 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1e11b4-bc25-47f1-b347-c16024b65abb admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:39.626887 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef538d26-d65c-41a7-a08e-720364530ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:39.627663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef538d26-d65c-41a7-a08e-720364530ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:39.627957 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef538d26-d65c-41a7-a08e-720364530ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:39.628233 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef538d26-d65c-41a7-a08e-720364530ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:39.630128 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef538d26-d65c-41a7-a08e-720364530ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:39.631133 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef538d26-d65c-41a7-a08e-720364530ca6 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:39.635625 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafa2d6-6432-4b8d-b83a-4e69418e04c4 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:39.642090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5109b42a-ce17-4197-9d19-d14c314e77d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:39.642829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109b42a-ce17-4197-9d19-d14c314e77d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:39.643169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109b42a-ce17-4197-9d19-d14c314e77d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:39.644699 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109b42a-ce17-4197-9d19-d14c314e77d3 admin admin] PATH_INFO: `/v3/projects/9db150ab0ee44ac6bb14ff2eab6fd197` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:39.644830 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1220/2433] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:39.645964 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5109b42a-ce17-4197-9d19-d14c314e77d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9db150ab0ee44ac6bb14ff2eab6fd197)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:39.648180 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5109b42a-ce17-4197-9d19-d14c314e77d3 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:39.648978 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef538d26-d65c-41a7-a08e-720364530ca6 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:39.663415 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef538d26-d65c-41a7-a08e-720364530ca6 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:39.668874 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb7dac-e10c-4bef-bfec-2dc075f97ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:39.670010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb7dac-e10c-4bef-bfec-2dc075f97ddd admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:39.670301 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb7dac-e10c-4bef-bfec-2dc075f97ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:39.670581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb7dac-e10c-4bef-bfec-2dc075f97ddd admin admin] PATH_INFO: `/v3/projects/46e2257762a34e65b114aa64f3253ad4/users/998220a0ea9944bdbf9269e514330d34/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:39.673744 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb7dac-e10c-4bef-bfec-2dc075f97ddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e2257762a34e65b114aa64f3253ad4, user_id=998220a0ea9944bdbf9269e514330d34, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:39.677690 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb7dac-e10c-4bef-bfec-2dc075f97ddd admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:39.696639 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1221/2434] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:39 2022] PUT /identity/v3/projects/46e2257762a34e65b114aa64f3253ad4/users/998220a0ea9944bdbf9269e514330d34/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:23:39.708092 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-5109b42a-ce17-4197-9d19-d14c314e77d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9db150ab0ee44ac6bb14ff2eab6fd197', 'request_id': 'req-5109b42a-ce17-4197-9d19-d14c314e77d3'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:39.717573 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-5109b42a-ce17-4197-9d19-d14c314e77d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9db150ab0ee44ac6bb14ff2eab6fd197', 'request_id': 'req-5109b42a-ce17-4197-9d19-d14c314e77d3'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:39.723668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-5109b42a-ce17-4197-9d19-d14c314e77d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9db150ab0ee44ac6bb14ff2eab6fd197', 'request_id': 'req-5109b42a-ce17-4197-9d19-d14c314e77d3'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:39.735520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-5109b42a-ce17-4197-9d19-d14c314e77d3 admin admin] The token cache is being invalidate because project 9db150ab0ee44ac6bb14ff2eab6fd197 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:23:39.735813 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-5109b42a-ce17-4197-9d19-d14c314e77d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:39.736494 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-5109b42a-ce17-4197-9d19-d14c314e77d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:39.738078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1214/2435] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:23:39 2022] DELETE /identity/v3/projects/9db150ab0ee44ac6bb14ff2eab6fd197 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:39.742589 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb7dac-e10c-4bef-bfec-2dc075f97ddd admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:39.744391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-6ecb7dac-e10c-4bef-bfec-2dc075f97ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:39.780517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:39.817476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb7dac-e10c-4bef-bfec-2dc075f97ddd admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:39.918100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45906679-d603-40e5-b12f-c86f366b6893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:39.919542 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45906679-d603-40e5-b12f-c86f366b6893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:39.919542 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45906679-d603-40e5-b12f-c86f366b6893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:39.920086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45906679-d603-40e5-b12f-c86f366b6893 admin admin] PATH_INFO: `/v3/users/a4089c3ebc6f4c3994480adcda065c19` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:39.922484 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45906679-d603-40e5-b12f-c86f366b6893 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4089c3ebc6f4c3994480adcda065c19)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:39.924858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45906679-d603-40e5-b12f-c86f366b6893 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:40.013048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-45906679-d603-40e5-b12f-c86f366b6893 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4089c3ebc6f4c3994480adcda065c19'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:40.024191 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-45906679-d603-40e5-b12f-c86f366b6893 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4089c3ebc6f4c3994480adcda065c19'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:40.025135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-45906679-d603-40e5-b12f-c86f366b6893 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4089c3ebc6f4c3994480adcda065c19'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:40.044063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-45906679-d603-40e5-b12f-c86f366b6893 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4089c3ebc6f4c3994480adcda065c19'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:40.054690 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-45906679-d603-40e5-b12f-c86f366b6893 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4089c3ebc6f4c3994480adcda065c19'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:40.076223 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1222/2436] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:23:39 2022] DELETE /identity/v3/users/a4089c3ebc6f4c3994480adcda065c19 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:40.650235 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75775e-8515-4418-835c-5cd26f1f05a2 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:40.650779 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75775e-8515-4418-835c-5cd26f1f05a2 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:40.650779 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d75775e-8515-4418-835c-5cd26f1f05a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:40.691392 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-6d75775e-8515-4418-835c-5cd26f1f05a2 None None] MFA Rules not processed for user `998220a0ea9944bdbf9269e514330d34`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:40.692761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-6d75775e-8515-4418-835c-5cd26f1f05a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:40.724662 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:40.800647 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1215/2437] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:40 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:40.931904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45906679-d603-40e5-b12f-c86f366b6893 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:40.939783 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-45906679-d603-40e5-b12f-c86f366b6893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:40.950381 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75775e-8515-4418-835c-5cd26f1f05a2 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:40.952150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-6d75775e-8515-4418-835c-5cd26f1f05a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:41.047936 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:41.055066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45906679-d603-40e5-b12f-c86f366b6893 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:41.055472 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d75775e-8515-4418-835c-5cd26f1f05a2 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:41.064544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa7fc8-e27d-44c2-8c73-24e3f5bd0ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:41.066244 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30c68f-992b-4847-853e-6cac7de5413c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:41.067410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa7fc8-e27d-44c2-8c73-24e3f5bd0ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:41.067867 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa7fc8-e27d-44c2-8c73-24e3f5bd0ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:41.068664 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30c68f-992b-4847-853e-6cac7de5413c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:41.068780 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa7fc8-e27d-44c2-8c73-24e3f5bd0ef7 admin admin] PATH_INFO: `/v3/users/11bf0abff172401bb62653e975c91f45` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:41.069650 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30c68f-992b-4847-853e-6cac7de5413c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:41.069932 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30c68f-992b-4847-853e-6cac7de5413c admin admin] PATH_INFO: `/v3/projects/b20c8651d8394dea9414bb617516b1fd` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:41.074310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa7fc8-e27d-44c2-8c73-24e3f5bd0ef7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11bf0abff172401bb62653e975c91f45)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:41.074310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa7fc8-e27d-44c2-8c73-24e3f5bd0ef7 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:41.077732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30c68f-992b-4847-853e-6cac7de5413c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b20c8651d8394dea9414bb617516b1fd)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:41.081587 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b30c68f-992b-4847-853e-6cac7de5413c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:41.136398 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3baa7fc8-e27d-44c2-8c73-24e3f5bd0ef7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11bf0abff172401bb62653e975c91f45'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.146323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3baa7fc8-e27d-44c2-8c73-24e3f5bd0ef7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11bf0abff172401bb62653e975c91f45'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.147292 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3baa7fc8-e27d-44c2-8c73-24e3f5bd0ef7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11bf0abff172401bb62653e975c91f45'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.154579 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1b30c68f-992b-4847-853e-6cac7de5413c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b20c8651d8394dea9414bb617516b1fd', 'request_id': 'req-1b30c68f-992b-4847-853e-6cac7de5413c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.165317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1b30c68f-992b-4847-853e-6cac7de5413c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b20c8651d8394dea9414bb617516b1fd', 'request_id': 'req-1b30c68f-992b-4847-853e-6cac7de5413c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.167122 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3baa7fc8-e27d-44c2-8c73-24e3f5bd0ef7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11bf0abff172401bb62653e975c91f45'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.168304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3baa7fc8-e27d-44c2-8c73-24e3f5bd0ef7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11bf0abff172401bb62653e975c91f45'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.174820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1b30c68f-992b-4847-853e-6cac7de5413c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b20c8651d8394dea9414bb617516b1fd', 'request_id': 'req-1b30c68f-992b-4847-853e-6cac7de5413c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.190813 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1223/2438] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:23:40 2022] DELETE /identity/v3/users/11bf0abff172401bb62653e975c91f45 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:41.191029 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1b30c68f-992b-4847-853e-6cac7de5413c admin admin] The token cache is being invalidate because project b20c8651d8394dea9414bb617516b1fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:23:41.191159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1b30c68f-992b-4847-853e-6cac7de5413c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.191805 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1b30c68f-992b-4847-853e-6cac7de5413c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.193336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1216/2439] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:23:40 2022] DELETE /identity/v3/projects/b20c8651d8394dea9414bb617516b1fd => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:23:41.198020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f115c-60f3-4211-8951-83022c640cbf None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:41.198849 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f115c-60f3-4211-8951-83022c640cbf None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:41.198849 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6f115c-60f3-4211-8951-83022c640cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:41.221226 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-af6f115c-60f3-4211-8951-83022c640cbf None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:41.222471 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-af6f115c-60f3-4211-8951-83022c640cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:41.287301 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1224/2440] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:23:41.295062 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d5f97-5887-49d0-b527-c57c728cfd35 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:41.295332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d5f97-5887-49d0-b527-c57c728cfd35 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:41.295540 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9d5f97-5887-49d0-b527-c57c728cfd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:41.323107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-ff9d5f97-5887-49d0-b527-c57c728cfd35 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:41.325433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ff9d5f97-5887-49d0-b527-c57c728cfd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:41.341987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1217/2441] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:41.346110 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f115c-60f3-4211-8951-83022c640cbf None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:41.357954 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6f115c-60f3-4211-8951-83022c640cbf None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:41.365573 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d385367-efd4-4290-92f1-d20bd3b65c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:41.366234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d385367-efd4-4290-92f1-d20bd3b65c94 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:41.366520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d385367-efd4-4290-92f1-d20bd3b65c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:41.366794 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d385367-efd4-4290-92f1-d20bd3b65c94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:41.367961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d385367-efd4-4290-92f1-d20bd3b65c94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:23:41.368279 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d385367-efd4-4290-92f1-d20bd3b65c94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:41.370202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d385367-efd4-4290-92f1-d20bd3b65c94 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:41.388802 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1225/2442] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:23:41.390969 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d5f97-5887-49d0-b527-c57c728cfd35 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:41.404179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9d5f97-5887-49d0-b527-c57c728cfd35 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:41.412924 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87de17-e66d-41fb-9210-a4a50f5bfb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:41.414018 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87de17-e66d-41fb-9210-a4a50f5bfb36 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:41.415273 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87de17-e66d-41fb-9210-a4a50f5bfb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:41.415986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87de17-e66d-41fb-9210-a4a50f5bfb36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:41.416438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87de17-e66d-41fb-9210-a4a50f5bfb36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:41.419222 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87de17-e66d-41fb-9210-a4a50f5bfb36 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:41.481153 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d385367-efd4-4290-92f1-d20bd3b65c94 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:41.495032 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1218/2443] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:23:41 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:41.504542 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d385367-efd4-4290-92f1-d20bd3b65c94 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:41.540567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87de17-e66d-41fb-9210-a4a50f5bfb36 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:41.542491 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-8a87de17-e66d-41fb-9210-a4a50f5bfb36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:41.574213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:41.617110 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3619c-54a6-4bec-bff7-8522a0f3b4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:41.617923 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3619c-54a6-4bec-bff7-8522a0f3b4eb admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:41.618214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3619c-54a6-4bec-bff7-8522a0f3b4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:41.618544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3619c-54a6-4bec-bff7-8522a0f3b4eb admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:41.619441 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d3619c-54a6-4bec-bff7-8522a0f3b4eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:41.621958 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d3619c-54a6-4bec-bff7-8522a0f3b4eb admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:41.622627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87de17-e66d-41fb-9210-a4a50f5bfb36 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:41.628687 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15addf1-e0a3-4bf1-887f-e07d5b278254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:41.629478 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15addf1-e0a3-4bf1-887f-e07d5b278254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:41.630103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15addf1-e0a3-4bf1-887f-e07d5b278254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:41.630450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15addf1-e0a3-4bf1-887f-e07d5b278254 admin admin] PATH_INFO: `/v3/projects/d7f54ee4ba204ceb982f8e4ad5dee102` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:41.631718 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15addf1-e0a3-4bf1-887f-e07d5b278254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7f54ee4ba204ceb982f8e4ad5dee102)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:41.633786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15addf1-e0a3-4bf1-887f-e07d5b278254 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:41.634584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:41.652948 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1226/2444] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:23:41 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:41.658643 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3619c-54a6-4bec-bff7-8522a0f3b4eb admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:41.675170 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d3619c-54a6-4bec-bff7-8522a0f3b4eb admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:41.680235 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7012d-35e2-48f7-b690-3180f4212c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:41.681183 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7012d-35e2-48f7-b690-3180f4212c5a admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:41.681329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7012d-35e2-48f7-b690-3180f4212c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:41.681609 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7012d-35e2-48f7-b690-3180f4212c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:41.682351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7012d-35e2-48f7-b690-3180f4212c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:41.684073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf7012d-35e2-48f7-b690-3180f4212c5a admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:41.700233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c15addf1-e0a3-4bf1-887f-e07d5b278254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7f54ee4ba204ceb982f8e4ad5dee102', 'request_id': 'req-c15addf1-e0a3-4bf1-887f-e07d5b278254'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.700870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1227/2445] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:41.701366 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7012d-35e2-48f7-b690-3180f4212c5a admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:41.709646 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c15addf1-e0a3-4bf1-887f-e07d5b278254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7f54ee4ba204ceb982f8e4ad5dee102', 'request_id': 'req-c15addf1-e0a3-4bf1-887f-e07d5b278254'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.712846 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7012d-35e2-48f7-b690-3180f4212c5a admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:41.719658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c15addf1-e0a3-4bf1-887f-e07d5b278254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7f54ee4ba204ceb982f8e4ad5dee102', 'request_id': 'req-c15addf1-e0a3-4bf1-887f-e07d5b278254'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.735642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c15addf1-e0a3-4bf1-887f-e07d5b278254 admin admin] The token cache is being invalidate because project d7f54ee4ba204ceb982f8e4ad5dee102 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:23:41.735917 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c15addf1-e0a3-4bf1-887f-e07d5b278254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.736594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c15addf1-e0a3-4bf1-887f-e07d5b278254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.738319 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1219/2446] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:23:41 2022] DELETE /identity/v3/projects/d7f54ee4ba204ceb982f8e4ad5dee102 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:41.742839 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15addf1-e0a3-4bf1-887f-e07d5b278254 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:41.744567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-c15addf1-e0a3-4bf1-887f-e07d5b278254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:41.784795 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aac095-0906-4c6c-b9ff-90429fa6c696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:41.786023 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aac095-0906-4c6c-b9ff-90429fa6c696 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:41.786306 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aac095-0906-4c6c-b9ff-90429fa6c696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:41.786650 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aac095-0906-4c6c-b9ff-90429fa6c696 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:41.787281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aac095-0906-4c6c-b9ff-90429fa6c696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:41.789440 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aac095-0906-4c6c-b9ff-90429fa6c696 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:41.801111 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1228/2447] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:23:41.805217 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aac095-0906-4c6c-b9ff-90429fa6c696 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:41.806878 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-26aac095-0906-4c6c-b9ff-90429fa6c696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:41.821019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aac095-0906-4c6c-b9ff-90429fa6c696 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:41.827035 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cdf155-7393-4091-8dca-725055175a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:41.827793 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdf155-7393-4091-8dca-725055175a8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:41.828067 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdf155-7393-4091-8dca-725055175a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:41.828341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdf155-7393-4091-8dca-725055175a8f admin admin] PATH_INFO: `/v3/projects/265a3a38ac2e4beda80f0569ec2dc90d/users/cef03a80972d40ffb9872878108bec11/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:41.832384 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15addf1-e0a3-4bf1-887f-e07d5b278254 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:41.839966 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04657c3e-b007-4ba6-8055-2d8d4723a620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:41.841172 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04657c3e-b007-4ba6-8055-2d8d4723a620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:41.841458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04657c3e-b007-4ba6-8055-2d8d4723a620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:41.841733 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04657c3e-b007-4ba6-8055-2d8d4723a620 admin admin] PATH_INFO: `/v3/users/56ee73ed8e244a4e8a434c4cd7f732e9` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:41.843056 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04657c3e-b007-4ba6-8055-2d8d4723a620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56ee73ed8e244a4e8a434c4cd7f732e9)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:41.845436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04657c3e-b007-4ba6-8055-2d8d4723a620 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:41.857429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cdf155-7393-4091-8dca-725055175a8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=265a3a38ac2e4beda80f0569ec2dc90d, user_id=cef03a80972d40ffb9872878108bec11, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:41.861935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cdf155-7393-4091-8dca-725055175a8f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:41.883374 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1229/2448] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:41 2022] PUT /identity/v3/projects/265a3a38ac2e4beda80f0569ec2dc90d/users/cef03a80972d40ffb9872878108bec11/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:23:41.895764 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7116910-53c6-4583-900c-ca0feb9fe604 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:41.896060 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7116910-53c6-4583-900c-ca0feb9fe604 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:41.896333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7116910-53c6-4583-900c-ca0feb9fe604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:41.930915 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-04657c3e-b007-4ba6-8055-2d8d4723a620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56ee73ed8e244a4e8a434c4cd7f732e9'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.932323 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-04657c3e-b007-4ba6-8055-2d8d4723a620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56ee73ed8e244a4e8a434c4cd7f732e9'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.933836 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-04657c3e-b007-4ba6-8055-2d8d4723a620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56ee73ed8e244a4e8a434c4cd7f732e9'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.946639 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:41.952344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-b7116910-53c6-4583-900c-ca0feb9fe604 None None] MFA Rules not processed for user `cef03a80972d40ffb9872878108bec11`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:41.953884 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b7116910-53c6-4583-900c-ca0feb9fe604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:41.955968 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-04657c3e-b007-4ba6-8055-2d8d4723a620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56ee73ed8e244a4e8a434c4cd7f732e9'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.965411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-04657c3e-b007-4ba6-8055-2d8d4723a620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56ee73ed8e244a4e8a434c4cd7f732e9'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:41.987908 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1220/2449] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:23:41 2022] DELETE /identity/v3/users/56ee73ed8e244a4e8a434c4cd7f732e9 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:23:42.033376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1230/2450] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:23:42.037987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04657c3e-b007-4ba6-8055-2d8d4723a620 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:42.039622 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-04657c3e-b007-4ba6-8055-2d8d4723a620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:42.122406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04657c3e-b007-4ba6-8055-2d8d4723a620 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:42.128482 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516463f-6519-4ec2-adb5-3d2f0c07f058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:42.129234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516463f-6519-4ec2-adb5-3d2f0c07f058 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:42.129506 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516463f-6519-4ec2-adb5-3d2f0c07f058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:42.129814 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1516463f-6519-4ec2-adb5-3d2f0c07f058 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:42.130580 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516463f-6519-4ec2-adb5-3d2f0c07f058 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:42.134091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1516463f-6519-4ec2-adb5-3d2f0c07f058 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:42.182837 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1221/2451] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:23:42 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:23:42.187615 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7116910-53c6-4583-900c-ca0feb9fe604 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:42.209487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516463f-6519-4ec2-adb5-3d2f0c07f058 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:42.213145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-1516463f-6519-4ec2-adb5-3d2f0c07f058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:42.259637 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:42.278953 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7116910-53c6-4583-900c-ca0feb9fe604 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:42.297084 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1516463f-6519-4ec2-adb5-3d2f0c07f058 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:42.302186 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe1d45-ec23-4864-8943-5f890627b82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:42.302817 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe1d45-ec23-4864-8943-5f890627b82c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:42.303089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe1d45-ec23-4864-8943-5f890627b82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:42.303358 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe1d45-ec23-4864-8943-5f890627b82c admin admin] PATH_INFO: `/v3/projects/14a7085aaa61410885bb0d39cb6d80b0` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:42.304807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe1d45-ec23-4864-8943-5f890627b82c admin admin] RBAC: Authorizing `identity:delete_project(project_id=14a7085aaa61410885bb0d39cb6d80b0)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:42.306757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fe1d45-ec23-4864-8943-5f890627b82c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:42.337408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00100c-9905-4005-854a-5e804a0d378f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:42.337968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00100c-9905-4005-854a-5e804a0d378f admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:42.338272 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00100c-9905-4005-854a-5e804a0d378f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:42.338587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d00100c-9905-4005-854a-5e804a0d378f admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:42.339321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00100c-9905-4005-854a-5e804a0d378f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:42.341273 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d00100c-9905-4005-854a-5e804a0d378f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:42.356814 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-30fe1d45-ec23-4864-8943-5f890627b82c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14a7085aaa61410885bb0d39cb6d80b0', 'request_id': 'req-30fe1d45-ec23-4864-8943-5f890627b82c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:42.365332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-30fe1d45-ec23-4864-8943-5f890627b82c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14a7085aaa61410885bb0d39cb6d80b0', 'request_id': 'req-30fe1d45-ec23-4864-8943-5f890627b82c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:42.365537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1231/2452] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:23:42 2022] POST /identity/v3/users => generated 298 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:42.366468 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00100c-9905-4005-854a-5e804a0d378f admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:42.368368 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-3d00100c-9905-4005-854a-5e804a0d378f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:42.370134 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:42.372592 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-30fe1d45-ec23-4864-8943-5f890627b82c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14a7085aaa61410885bb0d39cb6d80b0', 'request_id': 'req-30fe1d45-ec23-4864-8943-5f890627b82c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:42.380460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d00100c-9905-4005-854a-5e804a0d378f admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:42.387044 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-30fe1d45-ec23-4864-8943-5f890627b82c admin admin] The token cache is being invalidate because project 14a7085aaa61410885bb0d39cb6d80b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:23:42.387336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-30fe1d45-ec23-4864-8943-5f890627b82c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:42.387996 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-30fe1d45-ec23-4864-8943-5f890627b82c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:42.389566 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1222/2453] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:23:42 2022] DELETE /identity/v3/projects/14a7085aaa61410885bb0d39cb6d80b0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:23:42.393169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe1d45-ec23-4864-8943-5f890627b82c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:42.394539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-30fe1d45-ec23-4864-8943-5f890627b82c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:42.443698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f135ae-4879-478f-a733-01bb40caf9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:42.444367 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f135ae-4879-478f-a733-01bb40caf9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:42.444675 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f135ae-4879-478f-a733-01bb40caf9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:42.444950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f135ae-4879-478f-a733-01bb40caf9e8 admin admin] PATH_INFO: `/v3/users/29b3c639d0514b63b0f5976560e7663e` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:42.446170 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f135ae-4879-478f-a733-01bb40caf9e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29b3c639d0514b63b0f5976560e7663e)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:42.448100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f135ae-4879-478f-a733-01bb40caf9e8 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:42.462326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe1d45-ec23-4864-8943-5f890627b82c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:42.467211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1dac28-bae4-4103-a6cd-40b4df145326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:42.467841 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1dac28-bae4-4103-a6cd-40b4df145326 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:42.468122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1dac28-bae4-4103-a6cd-40b4df145326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:42.468926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1dac28-bae4-4103-a6cd-40b4df145326 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:42.469633 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1dac28-bae4-4103-a6cd-40b4df145326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:42.471264 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1dac28-bae4-4103-a6cd-40b4df145326 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:42.497185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1223/2454] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:42.500659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fa05a-2b48-4ffa-87f1-e98a654313b5 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:42.500659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fa05a-2b48-4ffa-87f1-e98a654313b5 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:42.500659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fa05a-2b48-4ffa-87f1-e98a654313b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:42.507166 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-21f135ae-4879-478f-a733-01bb40caf9e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29b3c639d0514b63b0f5976560e7663e'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:42.515412 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-21f135ae-4879-478f-a733-01bb40caf9e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29b3c639d0514b63b0f5976560e7663e'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:42.516144 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-21f135ae-4879-478f-a733-01bb40caf9e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29b3c639d0514b63b0f5976560e7663e'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:42.529208 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-21f135ae-4879-478f-a733-01bb40caf9e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29b3c639d0514b63b0f5976560e7663e'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:42.529932 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-21f135ae-4879-478f-a733-01bb40caf9e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29b3c639d0514b63b0f5976560e7663e'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:42.531427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-e62fa05a-2b48-4ffa-87f1-e98a654313b5 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:42.532917 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e62fa05a-2b48-4ffa-87f1-e98a654313b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:42.544940 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:42.545147 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1224/2455] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:42.545893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fa05a-2b48-4ffa-87f1-e98a654313b5 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:42.548065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e62fa05a-2b48-4ffa-87f1-e98a654313b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:42.551735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1232/2456] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:23:42 2022] DELETE /identity/v3/users/29b3c639d0514b63b0f5976560e7663e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:42.556300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e29b7-be85-4fdf-8c4e-d59dc1f8f35c None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:42.556737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e29b7-be85-4fdf-8c4e-d59dc1f8f35c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:42.557056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e29b7-be85-4fdf-8c4e-d59dc1f8f35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:42.564511 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fa05a-2b48-4ffa-87f1-e98a654313b5 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:42.591153 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-235e29b7-be85-4fdf-8c4e-d59dc1f8f35c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:42.592326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-235e29b7-be85-4fdf-8c4e-d59dc1f8f35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:42.661108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14de8b-8986-409d-ab21-6cd7b31233d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:42.662211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14de8b-8986-409d-ab21-6cd7b31233d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:42.662566 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14de8b-8986-409d-ab21-6cd7b31233d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:42.663297 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e14de8b-8986-409d-ab21-6cd7b31233d5 admin admin] PATH_INFO: `/v3/projects/74d58c1d4a1a45e09c588ac1633de287/users/96eaaba7ca134efaa26d13bc8df7cf6f/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:42.678869 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1233/2457] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:42.682781 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e29b7-be85-4fdf-8c4e-d59dc1f8f35c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:42.690773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14de8b-8986-409d-ab21-6cd7b31233d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d58c1d4a1a45e09c588ac1633de287, user_id=96eaaba7ca134efaa26d13bc8df7cf6f, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:42.693995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e29b7-be85-4fdf-8c4e-d59dc1f8f35c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:42.697542 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e14de8b-8986-409d-ab21-6cd7b31233d5 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:42.701288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5aee78-71d4-4d9e-986d-0be603525de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:42.702138 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5aee78-71d4-4d9e-986d-0be603525de6 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:42.703173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5aee78-71d4-4d9e-986d-0be603525de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:42.703387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5aee78-71d4-4d9e-986d-0be603525de6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:42.704200 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5aee78-71d4-4d9e-986d-0be603525de6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:23:42.704762 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5aee78-71d4-4d9e-986d-0be603525de6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:42.706789 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5aee78-71d4-4d9e-986d-0be603525de6 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:42.721347 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1225/2458] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:42 2022] PUT /identity/v3/projects/74d58c1d4a1a45e09c588ac1633de287/users/96eaaba7ca134efaa26d13bc8df7cf6f/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:42.725436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14de8b-8986-409d-ab21-6cd7b31233d5 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:42.726423 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1234/2459] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:23:42.731038 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5aee78-71d4-4d9e-986d-0be603525de6 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:42.751210 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5aee78-71d4-4d9e-986d-0be603525de6 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:42.817265 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e14de8b-8986-409d-ab21-6cd7b31233d5 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:42.823393 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921cf48-329e-44c0-a570-b05668cda8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:42.824111 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921cf48-329e-44c0-a570-b05668cda8a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:42.824355 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921cf48-329e-44c0-a570-b05668cda8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:42.824676 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921cf48-329e-44c0-a570-b05668cda8a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:42.825554 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921cf48-329e-44c0-a570-b05668cda8a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:42.828729 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921cf48-329e-44c0-a570-b05668cda8a2 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:42.831953 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:42.838525 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f214f-799c-443d-b060-19ef9de26d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:42.839352 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f214f-799c-443d-b060-19ef9de26d08 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:42.839737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f214f-799c-443d-b060-19ef9de26d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:42.840385 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930f214f-799c-443d-b060-19ef9de26d08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:42.841129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f214f-799c-443d-b060-19ef9de26d08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:42.843653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930f214f-799c-443d-b060-19ef9de26d08 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:42.855780 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1226/2460] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:23:42.857678 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921cf48-329e-44c0-a570-b05668cda8a2 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:42.864226 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d921cf48-329e-44c0-a570-b05668cda8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:42.885067 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921cf48-329e-44c0-a570-b05668cda8a2 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:42.890177 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10274d-cfa4-4a9f-bf9c-18b025621e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:42.890884 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10274d-cfa4-4a9f-bf9c-18b025621e55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:42.891181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10274d-cfa4-4a9f-bf9c-18b025621e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:42.891446 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10274d-cfa4-4a9f-bf9c-18b025621e55 admin admin] PATH_INFO: `/v3/projects/6e0e0182d3494f08a3b84b78740ae151` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:42.892666 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10274d-cfa4-4a9f-bf9c-18b025621e55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e0e0182d3494f08a3b84b78740ae151)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:42.895986 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1235/2461] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:23:42 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:42.895986 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f214f-799c-443d-b060-19ef9de26d08 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:42.896215 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10274d-cfa4-4a9f-bf9c-18b025621e55 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:42.951423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-2f10274d-cfa4-4a9f-bf9c-18b025621e55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e0e0182d3494f08a3b84b78740ae151', 'request_id': 'req-2f10274d-cfa4-4a9f-bf9c-18b025621e55'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:42.960809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-2f10274d-cfa4-4a9f-bf9c-18b025621e55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e0e0182d3494f08a3b84b78740ae151', 'request_id': 'req-2f10274d-cfa4-4a9f-bf9c-18b025621e55'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:42.967372 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-2f10274d-cfa4-4a9f-bf9c-18b025621e55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e0e0182d3494f08a3b84b78740ae151', 'request_id': 'req-2f10274d-cfa4-4a9f-bf9c-18b025621e55'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:42.971368 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930f214f-799c-443d-b060-19ef9de26d08 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:42.986418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:42.989694 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-2f10274d-cfa4-4a9f-bf9c-18b025621e55 admin admin] The token cache is being invalidate because project 6e0e0182d3494f08a3b84b78740ae151 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:23:42.993090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-2f10274d-cfa4-4a9f-bf9c-18b025621e55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:42.994269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-2f10274d-cfa4-4a9f-bf9c-18b025621e55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:42.996265 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1227/2462] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:23:42 2022] DELETE /identity/v3/projects/6e0e0182d3494f08a3b84b78740ae151 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:42.997828 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10274d-cfa4-4a9f-bf9c-18b025621e55 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:42.999599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2f10274d-cfa4-4a9f-bf9c-18b025621e55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:43.042571 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2dafcf-7761-487c-accc-88cf6a081b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:43.043281 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2dafcf-7761-487c-accc-88cf6a081b10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:43.043561 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2dafcf-7761-487c-accc-88cf6a081b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:43.043836 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2dafcf-7761-487c-accc-88cf6a081b10 admin admin] PATH_INFO: `/v3/projects/74d58c1d4a1a45e09c588ac1633de287/users/96eaaba7ca134efaa26d13bc8df7cf6f/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:43.046075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2dafcf-7761-487c-accc-88cf6a081b10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74d58c1d4a1a45e09c588ac1633de287, user_id=96eaaba7ca134efaa26d13bc8df7cf6f, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:43.050284 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2dafcf-7761-487c-accc-88cf6a081b10 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:43.072258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1236/2463] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:42 2022] PUT /identity/v3/projects/74d58c1d4a1a45e09c588ac1633de287/users/96eaaba7ca134efaa26d13bc8df7cf6f/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:43.072258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2dafcf-7761-487c-accc-88cf6a081b10 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:43.072258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-ed2dafcf-7761-487c-accc-88cf6a081b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:43.088910 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10274d-cfa4-4a9f-bf9c-18b025621e55 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:43.145682 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2dafcf-7761-487c-accc-88cf6a081b10 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:43.156814 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ab927-b774-444a-8991-a6208abced6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:43.157475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ab927-b774-444a-8991-a6208abced6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:43.157757 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ab927-b774-444a-8991-a6208abced6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:43.158036 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ab927-b774-444a-8991-a6208abced6a admin admin] PATH_INFO: `/v3/users/998220a0ea9944bdbf9269e514330d34` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:43.159311 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ab927-b774-444a-8991-a6208abced6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=998220a0ea9944bdbf9269e514330d34)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:43.160520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec5d40-c483-488a-b069-0c3efaa9563b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:43.161333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ab927-b774-444a-8991-a6208abced6a admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:43.161559 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec5d40-c483-488a-b069-0c3efaa9563b admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:43.161852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec5d40-c483-488a-b069-0c3efaa9563b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:43.162131 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec5d40-c483-488a-b069-0c3efaa9563b admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:43.162799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec5d40-c483-488a-b069-0c3efaa9563b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:43.164757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec5d40-c483-488a-b069-0c3efaa9563b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:43.190060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1228/2464] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:23:42 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:43.191670 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a9036-026b-45d1-8a88-e8bf737adb79 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:43.191959 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a9036-026b-45d1-8a88-e8bf737adb79 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:43.192231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a9036-026b-45d1-8a88-e8bf737adb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:43.222489 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-719a9036-026b-45d1-8a88-e8bf737adb79 None None] MFA Rules not processed for user `96eaaba7ca134efaa26d13bc8df7cf6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:43.223721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-719a9036-026b-45d1-8a88-e8bf737adb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:43.230075 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:43.238501 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-672ab927-b774-444a-8991-a6208abced6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '998220a0ea9944bdbf9269e514330d34'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:43.246956 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-672ab927-b774-444a-8991-a6208abced6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '998220a0ea9944bdbf9269e514330d34'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:43.248192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-672ab927-b774-444a-8991-a6208abced6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '998220a0ea9944bdbf9269e514330d34'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:43.268124 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-672ab927-b774-444a-8991-a6208abced6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '998220a0ea9944bdbf9269e514330d34'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:43.269505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-672ab927-b774-444a-8991-a6208abced6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '998220a0ea9944bdbf9269e514330d34'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:43.283897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1237/2465] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:23:43 2022] DELETE /identity/v3/users/998220a0ea9944bdbf9269e514330d34 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:43.287275 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ab927-b774-444a-8991-a6208abced6a admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:43.287275 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-672ab927-b774-444a-8991-a6208abced6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:43.317732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1229/2466] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:43.330379 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a9036-026b-45d1-8a88-e8bf737adb79 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:43.331621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:43.332538 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-719a9036-026b-45d1-8a88-e8bf737adb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:43.380418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ab927-b774-444a-8991-a6208abced6a admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:43.385226 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346d28e-007b-4c78-bb8b-ea2c99bbfc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:43.386446 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346d28e-007b-4c78-bb8b-ea2c99bbfc6b admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:43.386446 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346d28e-007b-4c78-bb8b-ea2c99bbfc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:43.386446 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346d28e-007b-4c78-bb8b-ea2c99bbfc6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:43.387307 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346d28e-007b-4c78-bb8b-ea2c99bbfc6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:43.389636 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346d28e-007b-4c78-bb8b-ea2c99bbfc6b admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:43.400675 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1238/2467] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:43.405380 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346d28e-007b-4c78-bb8b-ea2c99bbfc6b admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:43.415863 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346d28e-007b-4c78-bb8b-ea2c99bbfc6b admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:43.423238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a9036-026b-45d1-8a88-e8bf737adb79 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:43.424389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a815e3d2-d1e4-423d-9d92-babd72c844d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:43.425327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a815e3d2-d1e4-423d-9d92-babd72c844d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:43.425643 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a815e3d2-d1e4-423d-9d92-babd72c844d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:43.425921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a815e3d2-d1e4-423d-9d92-babd72c844d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:43.427110 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a815e3d2-d1e4-423d-9d92-babd72c844d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:43.429152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a815e3d2-d1e4-423d-9d92-babd72c844d9 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:43.444792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1239/2468] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:23:43.449664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a815e3d2-d1e4-423d-9d92-babd72c844d9 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:43.461604 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a815e3d2-d1e4-423d-9d92-babd72c844d9 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:43.469197 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6ee25-f352-4838-a9b8-8b4f4208650f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:43.470233 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6ee25-f352-4838-a9b8-8b4f4208650f admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:43.470687 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6ee25-f352-4838-a9b8-8b4f4208650f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:43.471514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd6ee25-f352-4838-a9b8-8b4f4208650f admin admin] PATH_INFO: `/v3/projects/5f4683550a9d40d1ab0439d002d73749/users/683f8f120df946f59505455a9ab2dc2a/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:43.503083 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6ee25-f352-4838-a9b8-8b4f4208650f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f4683550a9d40d1ab0439d002d73749, user_id=683f8f120df946f59505455a9ab2dc2a, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:43.506713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd6ee25-f352-4838-a9b8-8b4f4208650f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:43.510289 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32612861-813e-4104-9d42-1efe2707dfad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:43.510969 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32612861-813e-4104-9d42-1efe2707dfad service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:43.511294 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32612861-813e-4104-9d42-1efe2707dfad service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:43.511672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32612861-813e-4104-9d42-1efe2707dfad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:43.531730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1240/2469] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:43 2022] PUT /identity/v3/projects/5f4683550a9d40d1ab0439d002d73749/users/683f8f120df946f59505455a9ab2dc2a/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:43.531730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6ee25-f352-4838-a9b8-8b4f4208650f admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:43.541489 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-fbd6ee25-f352-4838-a9b8-8b4f4208650f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:43.598457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32612861-813e-4104-9d42-1efe2707dfad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:43.600066 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32612861-813e-4104-9d42-1efe2707dfad service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:43.611724 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd6ee25-f352-4838-a9b8-8b4f4208650f admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:43.614325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1230/2470] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:43 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:23:43.615486 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503ee18-f736-448d-ab97-f91ceb67d6c1 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:43.615845 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503ee18-f736-448d-ab97-f91ceb67d6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:43.616126 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503ee18-f736-448d-ab97-f91ceb67d6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:43.616716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb8b781-be67-4ce8-8569-006d97411174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:43.617448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8b781-be67-4ce8-8569-006d97411174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:43.617797 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8b781-be67-4ce8-8569-006d97411174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:43.618140 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb8b781-be67-4ce8-8569-006d97411174 admin admin] PATH_INFO: `/v3/projects/46e2257762a34e65b114aa64f3253ad4` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:43.619410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8b781-be67-4ce8-8569-006d97411174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46e2257762a34e65b114aa64f3253ad4)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:43.621935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb8b781-be67-4ce8-8569-006d97411174 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:43.645560 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-6503ee18-f736-448d-ab97-f91ceb67d6c1 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:43.647187 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-6503ee18-f736-448d-ab97-f91ceb67d6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:43.657082 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1231/2471] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:43.670783 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503ee18-f736-448d-ab97-f91ceb67d6c1 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:43.672977 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-6503ee18-f736-448d-ab97-f91ceb67d6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:43.688723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1eb8b781-be67-4ce8-8569-006d97411174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46e2257762a34e65b114aa64f3253ad4', 'request_id': 'req-1eb8b781-be67-4ce8-8569-006d97411174'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:43.689085 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:43.697667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1eb8b781-be67-4ce8-8569-006d97411174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46e2257762a34e65b114aa64f3253ad4', 'request_id': 'req-1eb8b781-be67-4ce8-8569-006d97411174'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:43.706805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1eb8b781-be67-4ce8-8569-006d97411174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46e2257762a34e65b114aa64f3253ad4', 'request_id': 'req-1eb8b781-be67-4ce8-8569-006d97411174'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:43.723088 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1eb8b781-be67-4ce8-8569-006d97411174 admin admin] The token cache is being invalidate because project 46e2257762a34e65b114aa64f3253ad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:23:43.723395 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1eb8b781-be67-4ce8-8569-006d97411174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:43.724176 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-1eb8b781-be67-4ce8-8569-006d97411174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:43.725818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1241/2472] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:23:43 2022] DELETE /identity/v3/projects/46e2257762a34e65b114aa64f3253ad4 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:43.731110 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bd098-6124-422e-be82-5043ded28688 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:43.731525 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bd098-6124-422e-be82-5043ded28688 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:43.732025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118bd098-6124-422e-be82-5043ded28688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:43.757790 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-118bd098-6124-422e-be82-5043ded28688 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:43.759284 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-118bd098-6124-422e-be82-5043ded28688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:43.763755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503ee18-f736-448d-ab97-f91ceb67d6c1 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:43.780321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:43.827232 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182edfa5-207d-4003-b36c-885b150c3cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:43.827837 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182edfa5-207d-4003-b36c-885b150c3cfe service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:43.828232 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182edfa5-207d-4003-b36c-885b150c3cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:43.828606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182edfa5-207d-4003-b36c-885b150c3cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:43.830580 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-182edfa5-207d-4003-b36c-885b150c3cfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:43.851623 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1242/2473] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:43.851623 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bd098-6124-422e-be82-5043ded28688 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:43.851623 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-118bd098-6124-422e-be82-5043ded28688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:43.915799 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118bd098-6124-422e-be82-5043ded28688 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:43.923470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe699c97-d50d-4dcf-b50e-a4c351b5171d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:43.924123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe699c97-d50d-4dcf-b50e-a4c351b5171d service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:43.924404 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe699c97-d50d-4dcf-b50e-a4c351b5171d service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:43.924907 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe699c97-d50d-4dcf-b50e-a4c351b5171d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:43.926927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-fe699c97-d50d-4dcf-b50e-a4c351b5171d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:43.927715 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182edfa5-207d-4003-b36c-885b150c3cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:43.929426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182edfa5-207d-4003-b36c-885b150c3cfe service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:43.942200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1232/2474] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:43.943915 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca2644-d62d-4312-8c28-e26a19393951 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:43.944413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca2644-d62d-4312-8c28-e26a19393951 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:43.944901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca2644-d62d-4312-8c28-e26a19393951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:43.969456 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-6cca2644-d62d-4312-8c28-e26a19393951 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:43.971036 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-6cca2644-d62d-4312-8c28-e26a19393951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:43.994216 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1233/2475] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:43.998534 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca2644-d62d-4312-8c28-e26a19393951 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:44.007784 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe699c97-d50d-4dcf-b50e-a4c351b5171d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:44.008784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca2644-d62d-4312-8c28-e26a19393951 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:44.009266 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe699c97-d50d-4dcf-b50e-a4c351b5171d service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:44.014003 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beba6fc-237d-4ee1-96f9-8ae653f07ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:44.014707 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beba6fc-237d-4ee1-96f9-8ae653f07ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:44.015109 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beba6fc-237d-4ee1-96f9-8ae653f07ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:44.015444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beba6fc-237d-4ee1-96f9-8ae653f07ff7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:44.016328 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beba6fc-237d-4ee1-96f9-8ae653f07ff7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:23:44.016908 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beba6fc-237d-4ee1-96f9-8ae653f07ff7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:44.018562 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beba6fc-237d-4ee1-96f9-8ae653f07ff7 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:44.022527 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1243/2476] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:43 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:44.036864 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1234/2477] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:23:44.038579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe699c97-d50d-4dcf-b50e-a4c351b5171d service cinder] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:44.054077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe699c97-d50d-4dcf-b50e-a4c351b5171d service cinder] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:44.059377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285cb8b-06e7-480e-9192-09f2747d82df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:44.059728 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285cb8b-06e7-480e-9192-09f2747d82df admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:44.060107 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285cb8b-06e7-480e-9192-09f2747d82df admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:44.060477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c285cb8b-06e7-480e-9192-09f2747d82df admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:44.061264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285cb8b-06e7-480e-9192-09f2747d82df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:44.063458 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c285cb8b-06e7-480e-9192-09f2747d82df admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:44.101289 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1244/2478] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:23:44 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:44.105391 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beba6fc-237d-4ee1-96f9-8ae653f07ff7 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:44.119245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beba6fc-237d-4ee1-96f9-8ae653f07ff7 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:44.178603 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:44.199520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec52552-05fa-46d0-a88d-c310a7c586ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:44.200357 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec52552-05fa-46d0-a88d-c310a7c586ac admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:44.200759 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec52552-05fa-46d0-a88d-c310a7c586ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:44.201083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec52552-05fa-46d0-a88d-c310a7c586ac admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:44.201761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec52552-05fa-46d0-a88d-c310a7c586ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:44.204350 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec52552-05fa-46d0-a88d-c310a7c586ac admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:44.238634 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1235/2479] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:23:44 2022] POST /identity/v3/users => generated 294 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:44.242807 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285cb8b-06e7-480e-9192-09f2747d82df admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:44.261528 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c285cb8b-06e7-480e-9192-09f2747d82df admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:44.270033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54338bae-56a2-4fbd-a4ae-bd54b3d41162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:44.270956 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54338bae-56a2-4fbd-a4ae-bd54b3d41162 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:44.271228 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54338bae-56a2-4fbd-a4ae-bd54b3d41162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:44.271504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54338bae-56a2-4fbd-a4ae-bd54b3d41162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:44.272207 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54338bae-56a2-4fbd-a4ae-bd54b3d41162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:44.274797 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54338bae-56a2-4fbd-a4ae-bd54b3d41162 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:44.292327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1245/2480] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:44.296051 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec52552-05fa-46d0-a88d-c310a7c586ac admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:44.311076 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec52552-05fa-46d0-a88d-c310a7c586ac admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:44.320237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a2616-6405-4fe7-acee-4ca2d185b069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:44.321137 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a2616-6405-4fe7-acee-4ca2d185b069 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:44.321420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a2616-6405-4fe7-acee-4ca2d185b069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:44.321689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a2616-6405-4fe7-acee-4ca2d185b069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:44.322536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a2616-6405-4fe7-acee-4ca2d185b069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:44.324162 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a2616-6405-4fe7-acee-4ca2d185b069 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:44.348407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1236/2481] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:44.350939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54338bae-56a2-4fbd-a4ae-bd54b3d41162 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:44.367327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54338bae-56a2-4fbd-a4ae-bd54b3d41162 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:44.380459 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fdf67-cc6e-47f9-94c4-939507f56856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:44.381244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fdf67-cc6e-47f9-94c4-939507f56856 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:44.381522 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fdf67-cc6e-47f9-94c4-939507f56856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:44.381823 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886fdf67-cc6e-47f9-94c4-939507f56856 admin admin] PATH_INFO: `/v3/projects/56eae010a28e44cb80f5ebc30d76490b/users/ba2f68dbcda24c03967c342a2cf9d23f/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:44.408293 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886fdf67-cc6e-47f9-94c4-939507f56856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56eae010a28e44cb80f5ebc30d76490b, user_id=ba2f68dbcda24c03967c342a2cf9d23f, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:44.411200 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886fdf67-cc6e-47f9-94c4-939507f56856 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:44.439510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1246/2482] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:44 2022] PUT /identity/v3/projects/56eae010a28e44cb80f5ebc30d76490b/users/ba2f68dbcda24c03967c342a2cf9d23f/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:44.449873 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc590c-9ed3-496a-9793-028778461869 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:44.450119 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc590c-9ed3-496a-9793-028778461869 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:44.450844 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc590c-9ed3-496a-9793-028778461869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:44.507006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-becc590c-9ed3-496a-9793-028778461869 None None] MFA Rules not processed for user `ba2f68dbcda24c03967c342a2cf9d23f`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:44.508213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-becc590c-9ed3-496a-9793-028778461869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:44.584887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1237/2483] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:44.590543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fdf67-cc6e-47f9-94c4-939507f56856 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:44.612482 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:44.658908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886fdf67-cc6e-47f9-94c4-939507f56856 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:44.663643 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770a43a-6506-4745-949a-8084aba70b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:44.664263 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770a43a-6506-4745-949a-8084aba70b6f admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:44.664539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770a43a-6506-4745-949a-8084aba70b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:44.664843 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c770a43a-6506-4745-949a-8084aba70b6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:44.665500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770a43a-6506-4745-949a-8084aba70b6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:44.667409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c770a43a-6506-4745-949a-8084aba70b6f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:44.698010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1247/2484] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:23:44 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:44.709631 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc590c-9ed3-496a-9793-028778461869 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:44.727747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc590c-9ed3-496a-9793-028778461869 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:44.805236 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f41d9-34e6-40b6-b5ad-1cd2f3b2456c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:44.806176 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f41d9-34e6-40b6-b5ad-1cd2f3b2456c admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:44.806572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f41d9-34e6-40b6-b5ad-1cd2f3b2456c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:44.808899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f41d9-34e6-40b6-b5ad-1cd2f3b2456c admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:44.808899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f41d9-34e6-40b6-b5ad-1cd2f3b2456c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:44.809800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f41d9-34e6-40b6-b5ad-1cd2f3b2456c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:44.841999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1238/2485] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:23:44 2022] POST /identity/v3/users => generated 299 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:44.846743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770a43a-6506-4745-949a-8084aba70b6f admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:44.858072 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c770a43a-6506-4745-949a-8084aba70b6f admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:44.863479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce3175-c48a-4fc0-bc67-e63ab1461cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:44.864855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce3175-c48a-4fc0-bc67-e63ab1461cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:44.865134 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce3175-c48a-4fc0-bc67-e63ab1461cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:44.865401 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce3175-c48a-4fc0-bc67-e63ab1461cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:44.866433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce3175-c48a-4fc0-bc67-e63ab1461cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:44.868143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce3175-c48a-4fc0-bc67-e63ab1461cc3 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:44.883107 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1248/2486] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:44.887510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f41d9-34e6-40b6-b5ad-1cd2f3b2456c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:44.898539 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f41d9-34e6-40b6-b5ad-1cd2f3b2456c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:44.903933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e0493-5c8b-4444-8d5c-9bda2093caa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:44.904665 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e0493-5c8b-4444-8d5c-9bda2093caa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:44.904941 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e0493-5c8b-4444-8d5c-9bda2093caa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:44.905225 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7e0493-5c8b-4444-8d5c-9bda2093caa9 admin admin] PATH_INFO: `/v3/projects/3ed51048805f443ebfad1b7ba832c591/users/62fb563101934e83b6d03f69ba805586/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:44.924085 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e0493-5c8b-4444-8d5c-9bda2093caa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ed51048805f443ebfad1b7ba832c591, user_id=62fb563101934e83b6d03f69ba805586, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:44.926798 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7e0493-5c8b-4444-8d5c-9bda2093caa9 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:44.944242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1239/2487] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:44 2022] PUT /identity/v3/projects/3ed51048805f443ebfad1b7ba832c591/users/62fb563101934e83b6d03f69ba805586/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:44.948527 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce3175-c48a-4fc0-bc67-e63ab1461cc3 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:44.960091 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce3175-c48a-4fc0-bc67-e63ab1461cc3 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:45.062496 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8016764b-633e-4835-b9c1-fc8b2b46e0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:45.063251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016764b-633e-4835-b9c1-fc8b2b46e0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:45.063747 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016764b-633e-4835-b9c1-fc8b2b46e0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:45.064141 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8016764b-633e-4835-b9c1-fc8b2b46e0a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:45.064953 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016764b-633e-4835-b9c1-fc8b2b46e0a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:45.066950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8016764b-633e-4835-b9c1-fc8b2b46e0a8 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:45.086860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1249/2488] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:23:45.088887 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e0493-5c8b-4444-8d5c-9bda2093caa9 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:45.104138 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7e0493-5c8b-4444-8d5c-9bda2093caa9 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:45.113535 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276f1f5-ee25-4eba-a851-2fc5b9e5c4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:45.115292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276f1f5-ee25-4eba-a851-2fc5b9e5c4f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:45.116117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276f1f5-ee25-4eba-a851-2fc5b9e5c4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:45.116473 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276f1f5-ee25-4eba-a851-2fc5b9e5c4f5 admin admin] PATH_INFO: `/v3/projects/3ed51048805f443ebfad1b7ba832c591/users/62fb563101934e83b6d03f69ba805586/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:45.121175 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276f1f5-ee25-4eba-a851-2fc5b9e5c4f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ed51048805f443ebfad1b7ba832c591, user_id=62fb563101934e83b6d03f69ba805586, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:45.123922 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276f1f5-ee25-4eba-a851-2fc5b9e5c4f5 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:45.141381 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1240/2489] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:45 2022] PUT /identity/v3/projects/3ed51048805f443ebfad1b7ba832c591/users/62fb563101934e83b6d03f69ba805586/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:23:45.151885 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f6e6a-5bc1-4534-b006-6d1019a28fdb None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:45.152126 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f6e6a-5bc1-4534-b006-6d1019a28fdb None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:45.152368 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f6e6a-5bc1-4534-b006-6d1019a28fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:45.182325 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c82f6e6a-5bc1-4534-b006-6d1019a28fdb None None] MFA Rules not processed for user `62fb563101934e83b6d03f69ba805586`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:45.183632 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c82f6e6a-5bc1-4534-b006-6d1019a28fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:45.274726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1250/2490] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:45.285523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276f1f5-ee25-4eba-a851-2fc5b9e5c4f5 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:45.298858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:45.366837 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276f1f5-ee25-4eba-a851-2fc5b9e5c4f5 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:45.370602 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a90a5-cef0-4087-a3d5-667c95104eca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:45.371260 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a90a5-cef0-4087-a3d5-667c95104eca service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:45.371543 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a90a5-cef0-4087-a3d5-667c95104eca service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:45.371820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a90a5-cef0-4087-a3d5-667c95104eca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:45.382280 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a90a5-cef0-4087-a3d5-667c95104eca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:45.383441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a90a5-cef0-4087-a3d5-667c95104eca service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:45.395875 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1241/2491] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:45.428975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f6e6a-5bc1-4534-b006-6d1019a28fdb None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:45.455082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:45.564717 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f6e6a-5bc1-4534-b006-6d1019a28fdb None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:45.564717 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03a0f8-f06a-4892-aacc-7e22e5040f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:45.564717 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03a0f8-f06a-4892-aacc-7e22e5040f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:45.564717 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03a0f8-f06a-4892-aacc-7e22e5040f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:45.565187 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e03a0f8-f06a-4892-aacc-7e22e5040f33 admin admin] PATH_INFO: `/v3/users/ba2f68dbcda24c03967c342a2cf9d23f` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:45.565862 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03a0f8-f06a-4892-aacc-7e22e5040f33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba2f68dbcda24c03967c342a2cf9d23f)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:45.567798 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e03a0f8-f06a-4892-aacc-7e22e5040f33 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:45.691524 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8e03a0f8-f06a-4892-aacc-7e22e5040f33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba2f68dbcda24c03967c342a2cf9d23f'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:45.717346 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8e03a0f8-f06a-4892-aacc-7e22e5040f33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba2f68dbcda24c03967c342a2cf9d23f'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:45.718780 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8e03a0f8-f06a-4892-aacc-7e22e5040f33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba2f68dbcda24c03967c342a2cf9d23f'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:45.737897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8e03a0f8-f06a-4892-aacc-7e22e5040f33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba2f68dbcda24c03967c342a2cf9d23f'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:45.738664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8e03a0f8-f06a-4892-aacc-7e22e5040f33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba2f68dbcda24c03967c342a2cf9d23f'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:45.779336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1251/2492] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:23:45 2022] DELETE /identity/v3/users/ba2f68dbcda24c03967c342a2cf9d23f => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:45.783743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701ed9c-524e-4005-816e-3c679c0a2703 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:45.784172 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701ed9c-524e-4005-816e-3c679c0a2703 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:45.784566 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701ed9c-524e-4005-816e-3c679c0a2703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:45.839470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-f701ed9c-524e-4005-816e-3c679c0a2703 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:45.847508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-f701ed9c-524e-4005-816e-3c679c0a2703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:45.990916 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1242/2493] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:46.013614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03a0f8-f06a-4892-aacc-7e22e5040f33 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:46.015197 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8e03a0f8-f06a-4892-aacc-7e22e5040f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:46.097308 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e03a0f8-f06a-4892-aacc-7e22e5040f33 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:46.113559 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f66f49-90e4-426e-9809-72307ee1c86c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:46.114493 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f66f49-90e4-426e-9809-72307ee1c86c service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:46.114940 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f66f49-90e4-426e-9809-72307ee1c86c service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:46.115369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f66f49-90e4-426e-9809-72307ee1c86c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:46.126632 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f66f49-90e4-426e-9809-72307ee1c86c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:46.128670 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f66f49-90e4-426e-9809-72307ee1c86c service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:46.140868 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1252/2494] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:46.920504 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701ed9c-524e-4005-816e-3c679c0a2703 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:46.924950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-f701ed9c-524e-4005-816e-3c679c0a2703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:47.004646 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701ed9c-524e-4005-816e-3c679c0a2703 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:47.015828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f66f49-90e4-426e-9809-72307ee1c86c service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:47.017273 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-78f66f49-90e4-426e-9809-72307ee1c86c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:47.033174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:47.035129 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:47.035515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:47.035892 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43 admin admin] PATH_INFO: `/v3/projects/56eae010a28e44cb80f5ebc30d76490b` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:47.037353 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56eae010a28e44cb80f5ebc30d76490b)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:47.039245 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f66f49-90e4-426e-9809-72307ee1c86c service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:47.039607 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:47.055223 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9eb90-e205-4fca-b19f-6530a08c8ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:47.055890 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9eb90-e205-4fca-b19f-6530a08c8ee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:47.056376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9eb90-e205-4fca-b19f-6530a08c8ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:47.056608 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9eb90-e205-4fca-b19f-6530a08c8ee8 admin admin] PATH_INFO: `/v3/users/cef03a80972d40ffb9872878108bec11` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:47.059575 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9eb90-e205-4fca-b19f-6530a08c8ee8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cef03a80972d40ffb9872878108bec11)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:47.065717 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9eb90-e205-4fca-b19f-6530a08c8ee8 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:47.129040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:47.155267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4ec9eb90-e205-4fca-b19f-6530a08c8ee8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cef03a80972d40ffb9872878108bec11'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:47.162825 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56eae010a28e44cb80f5ebc30d76490b', 'request_id': 'req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:47.179546 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4ec9eb90-e205-4fca-b19f-6530a08c8ee8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cef03a80972d40ffb9872878108bec11'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:47.180032 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56eae010a28e44cb80f5ebc30d76490b', 'request_id': 'req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:47.180932 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4ec9eb90-e205-4fca-b19f-6530a08c8ee8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cef03a80972d40ffb9872878108bec11'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:47.207361 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56eae010a28e44cb80f5ebc30d76490b', 'request_id': 'req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:47.223694 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43 admin admin] The token cache is being invalidate because project 56eae010a28e44cb80f5ebc30d76490b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:23:47.223993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:47.224684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:47.229069 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1243/2495] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:23:46 2022] DELETE /identity/v3/projects/56eae010a28e44cb80f5ebc30d76490b => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:47.230121 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:47.231932 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:47.241669 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4ec9eb90-e205-4fca-b19f-6530a08c8ee8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cef03a80972d40ffb9872878108bec11'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:47.242892 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4ec9eb90-e205-4fca-b19f-6530a08c8ee8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cef03a80972d40ffb9872878108bec11'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:47.273614 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1253/2496] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:23:47 2022] DELETE /identity/v3/users/cef03a80972d40ffb9872878108bec11 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:47.283260 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7921f-cab5-446c-874b-80cc4e3d0b66 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:47.283642 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7921f-cab5-446c-874b-80cc4e3d0b66 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:47.284098 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7921f-cab5-446c-874b-80cc4e3d0b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:47.328827 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-34c7921f-cab5-446c-874b-80cc4e3d0b66 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:47.330077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-34c7921f-cab5-446c-874b-80cc4e3d0b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:47.335228 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:47.390596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ae1e3-fce6-4b30-a3b6-ab92d9c39c43 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:47.443054 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1254/2497] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:47.454538 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7921f-cab5-446c-874b-80cc4e3d0b66 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:47.459172 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-34c7921f-cab5-446c-874b-80cc4e3d0b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:47.501444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704a8cf-50ca-4e70-92fb-75df3409443b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:47.502268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704a8cf-50ca-4e70-92fb-75df3409443b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:47.502549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704a8cf-50ca-4e70-92fb-75df3409443b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:47.502824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704a8cf-50ca-4e70-92fb-75df3409443b admin admin] PATH_INFO: `/v3/users/62fb563101934e83b6d03f69ba805586` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:47.504666 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704a8cf-50ca-4e70-92fb-75df3409443b admin admin] RBAC: Authorizing `identity:delete_user(user_id=62fb563101934e83b6d03f69ba805586)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:47.507256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704a8cf-50ca-4e70-92fb-75df3409443b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:47.563417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7921f-cab5-446c-874b-80cc4e3d0b66 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:47.568761 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3000c6a-870e-4238-a56c-d80388187b65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:47.569679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3000c6a-870e-4238-a56c-d80388187b65 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:47.570151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3000c6a-870e-4238-a56c-d80388187b65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:47.570627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3000c6a-870e-4238-a56c-d80388187b65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:47.582697 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3000c6a-870e-4238-a56c-d80388187b65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:47.584001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4704a8cf-50ca-4e70-92fb-75df3409443b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62fb563101934e83b6d03f69ba805586'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:47.584966 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4704a8cf-50ca-4e70-92fb-75df3409443b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62fb563101934e83b6d03f69ba805586'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:47.585274 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3000c6a-870e-4238-a56c-d80388187b65 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:47.585588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4704a8cf-50ca-4e70-92fb-75df3409443b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62fb563101934e83b6d03f69ba805586'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:47.601873 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4704a8cf-50ca-4e70-92fb-75df3409443b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62fb563101934e83b6d03f69ba805586'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:47.621057 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1255/2498] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:47.629494 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4704a8cf-50ca-4e70-92fb-75df3409443b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62fb563101934e83b6d03f69ba805586'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:47.679079 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1244/2499] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:23:47 2022] DELETE /identity/v3/users/62fb563101934e83b6d03f69ba805586 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:48.554996 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3000c6a-870e-4238-a56c-d80388187b65 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:48.557967 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d3000c6a-870e-4238-a56c-d80388187b65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:48.588169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3000c6a-870e-4238-a56c-d80388187b65 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:48.626972 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704a8cf-50ca-4e70-92fb-75df3409443b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:48.628527 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-4704a8cf-50ca-4e70-92fb-75df3409443b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:48.740985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:48.764608 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:48.765858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:48.766174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:48.769525 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08 admin admin] PATH_INFO: `/v3/projects/265a3a38ac2e4beda80f0569ec2dc90d` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:48.772685 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=265a3a38ac2e4beda80f0569ec2dc90d)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:48.781671 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:48.855395 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704a8cf-50ca-4e70-92fb-75df3409443b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:48.869743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386f928-7ea5-44c4-893d-b12e032ba58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:48.871169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386f928-7ea5-44c4-893d-b12e032ba58e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:48.871453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386f928-7ea5-44c4-893d-b12e032ba58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:48.871731 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386f928-7ea5-44c4-893d-b12e032ba58e admin admin] PATH_INFO: `/v3/projects/3ed51048805f443ebfad1b7ba832c591` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:48.877876 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386f928-7ea5-44c4-893d-b12e032ba58e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ed51048805f443ebfad1b7ba832c591)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:48.885727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9386f928-7ea5-44c4-893d-b12e032ba58e admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:48.902632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:48.917290 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '265a3a38ac2e4beda80f0569ec2dc90d', 'request_id': 'req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:48.939634 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '265a3a38ac2e4beda80f0569ec2dc90d', 'request_id': 'req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:48.961166 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '265a3a38ac2e4beda80f0569ec2dc90d', 'request_id': 'req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:48.983761 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08 admin admin] The token cache is being invalidate because project 265a3a38ac2e4beda80f0569ec2dc90d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:23:48.985362 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:48.989122 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:49.001376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1256/2500] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:23:48 2022] DELETE /identity/v3/projects/265a3a38ac2e4beda80f0569ec2dc90d => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:49.007186 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:49.009283 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:49.047731 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-9386f928-7ea5-44c4-893d-b12e032ba58e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ed51048805f443ebfad1b7ba832c591', 'request_id': 'req-9386f928-7ea5-44c4-893d-b12e032ba58e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:49.071212 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-9386f928-7ea5-44c4-893d-b12e032ba58e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ed51048805f443ebfad1b7ba832c591', 'request_id': 'req-9386f928-7ea5-44c4-893d-b12e032ba58e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:49.086894 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-9386f928-7ea5-44c4-893d-b12e032ba58e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ed51048805f443ebfad1b7ba832c591', 'request_id': 'req-9386f928-7ea5-44c4-893d-b12e032ba58e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:49.115655 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-9386f928-7ea5-44c4-893d-b12e032ba58e admin admin] The token cache is being invalidate because project 3ed51048805f443ebfad1b7ba832c591 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:23:49.115953 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-9386f928-7ea5-44c4-893d-b12e032ba58e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:49.119233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-9386f928-7ea5-44c4-893d-b12e032ba58e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:49.121897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1245/2501] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:23:48 2022] DELETE /identity/v3/projects/3ed51048805f443ebfad1b7ba832c591 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:49.128138 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe9704-6d74-4043-9fc1-612139ee2298 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:49.128421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe9704-6d74-4043-9fc1-612139ee2298 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:49.128736 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe9704-6d74-4043-9fc1-612139ee2298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:49.174312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-76fe9704-6d74-4043-9fc1-612139ee2298 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:49.178181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-76fe9704-6d74-4043-9fc1-612139ee2298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:49.191456 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3adbf-8055-4840-b48a-ac78c9d2fb08 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:49.330927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1246/2502] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:49.333934 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b7ae3-8ba5-4c58-88fe-0893079bef14 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:49.334236 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b7ae3-8ba5-4c58-88fe-0893079bef14 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:49.334514 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b7ae3-8ba5-4c58-88fe-0893079bef14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:49.335997 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f2cf0-0116-4404-b411-f710f1de9008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:49.337961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f2cf0-0116-4404-b411-f710f1de9008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:49.338246 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f2cf0-0116-4404-b411-f710f1de9008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:49.339652 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f2cf0-0116-4404-b411-f710f1de9008 admin admin] PATH_INFO: `/v3/users/96eaaba7ca134efaa26d13bc8df7cf6f` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:49.341389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f2cf0-0116-4404-b411-f710f1de9008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96eaaba7ca134efaa26d13bc8df7cf6f)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:49.346297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f2cf0-0116-4404-b411-f710f1de9008 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:49.388860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-d08b7ae3-8ba5-4c58-88fe-0893079bef14 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:49.393174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d08b7ae3-8ba5-4c58-88fe-0893079bef14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:49.440467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1247/2503] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:49.442543 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b7ae3-8ba5-4c58-88fe-0893079bef14 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:49.466749 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:49.485059 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b7ae3-8ba5-4c58-88fe-0893079bef14 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:49.497050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12392c5-c7dd-4f2c-aa69-0d1b66218056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:49.497991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12392c5-c7dd-4f2c-aa69-0d1b66218056 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:49.498642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12392c5-c7dd-4f2c-aa69-0d1b66218056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:49.499560 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12392c5-c7dd-4f2c-aa69-0d1b66218056 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:49.502005 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12392c5-c7dd-4f2c-aa69-0d1b66218056 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:23:49.502631 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-876f2cf0-0116-4404-b411-f710f1de9008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96eaaba7ca134efaa26d13bc8df7cf6f'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:49.504490 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12392c5-c7dd-4f2c-aa69-0d1b66218056 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:49.506359 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12392c5-c7dd-4f2c-aa69-0d1b66218056 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:49.519730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-876f2cf0-0116-4404-b411-f710f1de9008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96eaaba7ca134efaa26d13bc8df7cf6f'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:49.521707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-876f2cf0-0116-4404-b411-f710f1de9008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96eaaba7ca134efaa26d13bc8df7cf6f'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:49.540480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1248/2504] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:23:49.541368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12392c5-c7dd-4f2c-aa69-0d1b66218056 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:49.564345 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-876f2cf0-0116-4404-b411-f710f1de9008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96eaaba7ca134efaa26d13bc8df7cf6f'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:49.567337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-876f2cf0-0116-4404-b411-f710f1de9008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96eaaba7ca134efaa26d13bc8df7cf6f'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:49.576874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12392c5-c7dd-4f2c-aa69-0d1b66218056 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:49.588778 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45bf09-0028-43ca-849e-cc3f9f040896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:49.591222 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45bf09-0028-43ca-849e-cc3f9f040896 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:49.594204 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1257/2505] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:23:49 2022] DELETE /identity/v3/users/96eaaba7ca134efaa26d13bc8df7cf6f => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:49.594554 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45bf09-0028-43ca-849e-cc3f9f040896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:49.594998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45bf09-0028-43ca-849e-cc3f9f040896 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:49.595906 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e45bf09-0028-43ca-849e-cc3f9f040896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:49.612164 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e45bf09-0028-43ca-849e-cc3f9f040896 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:49.677604 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1249/2506] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:23:49 2022] POST /identity/v3/projects => generated 401 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:49.681600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f2cf0-0116-4404-b411-f710f1de9008 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:49.684250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-876f2cf0-0116-4404-b411-f710f1de9008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:49.794343 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f2cf0-0116-4404-b411-f710f1de9008 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:49.898573 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1c3f9-aa9e-4a8a-bb7d-e5383a3a591c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:49.900610 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1c3f9-aa9e-4a8a-bb7d-e5383a3a591c admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:49.901113 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1c3f9-aa9e-4a8a-bb7d-e5383a3a591c admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:49.901608 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1c3f9-aa9e-4a8a-bb7d-e5383a3a591c admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:49.902703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1c3f9-aa9e-4a8a-bb7d-e5383a3a591c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:49.905164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1c3f9-aa9e-4a8a-bb7d-e5383a3a591c admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:49.934488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1258/2507] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:23:49 2022] POST /identity/v3/users => generated 301 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:49.939299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45bf09-0028-43ca-849e-cc3f9f040896 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:49.952824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45bf09-0028-43ca-849e-cc3f9f040896 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:49.959583 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947e1f6-f48e-4410-9d53-fd2ef3d2c8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:49.960721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947e1f6-f48e-4410-9d53-fd2ef3d2c8ce admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:49.961107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947e1f6-f48e-4410-9d53-fd2ef3d2c8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:49.961483 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6947e1f6-f48e-4410-9d53-fd2ef3d2c8ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:49.962302 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947e1f6-f48e-4410-9d53-fd2ef3d2c8ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:49.964210 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6947e1f6-f48e-4410-9d53-fd2ef3d2c8ce admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:49.981882 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1250/2508] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:49.983214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1c3f9-aa9e-4a8a-bb7d-e5383a3a591c admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:50.009255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1c3f9-aa9e-4a8a-bb7d-e5383a3a591c admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:50.025532 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0f1ef-c4cc-44a4-ab2c-74bf2784175a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:50.027287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0f1ef-c4cc-44a4-ab2c-74bf2784175a admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:50.027930 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0f1ef-c4cc-44a4-ab2c-74bf2784175a admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:50.028387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0f1ef-c4cc-44a4-ab2c-74bf2784175a admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:50.029474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0f1ef-c4cc-44a4-ab2c-74bf2784175a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:50.031955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0f1ef-c4cc-44a4-ab2c-74bf2784175a admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:50.057885 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1259/2509] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:50.059181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947e1f6-f48e-4410-9d53-fd2ef3d2c8ce admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:50.071415 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6947e1f6-f48e-4410-9d53-fd2ef3d2c8ce admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:50.076852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef148acb-b271-419b-8553-5314b6f2693d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:50.078311 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef148acb-b271-419b-8553-5314b6f2693d admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:50.079251 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef148acb-b271-419b-8553-5314b6f2693d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:50.079251 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef148acb-b271-419b-8553-5314b6f2693d admin admin] PATH_INFO: `/v3/projects/bff29fe8fd7f4a8e9d05a0f8e4ab839d/users/9d9dbc4b93504f9eab5fb7425d9ce3f1/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:50.100268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef148acb-b271-419b-8553-5314b6f2693d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff29fe8fd7f4a8e9d05a0f8e4ab839d, user_id=9d9dbc4b93504f9eab5fb7425d9ce3f1, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:50.103710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef148acb-b271-419b-8553-5314b6f2693d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:50.125220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1251/2510] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:50 2022] PUT /identity/v3/projects/bff29fe8fd7f4a8e9d05a0f8e4ab839d/users/9d9dbc4b93504f9eab5fb7425d9ce3f1/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:50.136030 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62ca0c-d425-4fc1-a0ee-bce905bfe8de None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:50.136331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62ca0c-d425-4fc1-a0ee-bce905bfe8de None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:50.136591 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff62ca0c-d425-4fc1-a0ee-bce905bfe8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:50.189420 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-ff62ca0c-d425-4fc1-a0ee-bce905bfe8de None None] MFA Rules not processed for user `9d9dbc4b93504f9eab5fb7425d9ce3f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:50.191146 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-ff62ca0c-d425-4fc1-a0ee-bce905bfe8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:50.225947 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:50.341705 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1260/2511] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:50 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:50.343045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef148acb-b271-419b-8553-5314b6f2693d admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:50.404825 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:50.505778 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef148acb-b271-419b-8553-5314b6f2693d admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:50.513942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba2b01-e557-40fa-9f95-de0f0633dcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:50.514716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba2b01-e557-40fa-9f95-de0f0633dcc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:50.515764 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba2b01-e557-40fa-9f95-de0f0633dcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:50.516112 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba2b01-e557-40fa-9f95-de0f0633dcc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:50.517177 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ba2b01-e557-40fa-9f95-de0f0633dcc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:50.526482 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ba2b01-e557-40fa-9f95-de0f0633dcc3 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:50.570813 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1252/2512] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:23:50 2022] POST /identity/v3/projects => generated 401 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:50.572251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62ca0c-d425-4fc1-a0ee-bce905bfe8de None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:50.594494 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff62ca0c-d425-4fc1-a0ee-bce905bfe8de None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:50.629986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b65b9c-b9a0-424a-a45d-8a7bbc4cc064 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:50.630269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b65b9c-b9a0-424a-a45d-8a7bbc4cc064 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:50.630544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b65b9c-b9a0-424a-a45d-8a7bbc4cc064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:50.688037 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-f5b65b9c-b9a0-424a-a45d-8a7bbc4cc064 None None] MFA Rules not processed for user `683f8f120df946f59505455a9ab2dc2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:50.689073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-f5b65b9c-b9a0-424a-a45d-8a7bbc4cc064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:50.750259 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f7ae9-277d-4c06-952b-5cb8476f7300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:50.750893 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f7ae9-277d-4c06-952b-5cb8476f7300 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:50.751676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f7ae9-277d-4c06-952b-5cb8476f7300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:50.751676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f7ae9-277d-4c06-952b-5cb8476f7300 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:50.752100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f7ae9-277d-4c06-952b-5cb8476f7300 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:50.754244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f7ae9-277d-4c06-952b-5cb8476f7300 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:50.791674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1261/2513] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:23:50 2022] POST /identity/v3/users => generated 307 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:50.791674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f7ae9-277d-4c06-952b-5cb8476f7300 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:50.793253 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d61f7ae9-277d-4c06-952b-5cb8476f7300 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:50.809285 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f7ae9-277d-4c06-952b-5cb8476f7300 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:50.816610 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:50.817417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:50.818559 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:50.819002 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d admin admin] PATH_INFO: `/v3/projects/74d58c1d4a1a45e09c588ac1633de287` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:50.820462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d admin admin] RBAC: Authorizing `identity:delete_project(project_id=74d58c1d4a1a45e09c588ac1633de287)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:50.825124 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:50.825668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1253/2514] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:23:50.827143 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b65b9c-b9a0-424a-a45d-8a7bbc4cc064 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:50.860163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b65b9c-b9a0-424a-a45d-8a7bbc4cc064 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:50.869603 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649e5b8-2d41-4d50-a722-b0285338cdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:50.870594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649e5b8-2d41-4d50-a722-b0285338cdcd admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:50.876725 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649e5b8-2d41-4d50-a722-b0285338cdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:50.877363 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649e5b8-2d41-4d50-a722-b0285338cdcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:50.878756 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649e5b8-2d41-4d50-a722-b0285338cdcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:50.882267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649e5b8-2d41-4d50-a722-b0285338cdcd admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:50.898065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1254/2515] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:50.899556 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649e5b8-2d41-4d50-a722-b0285338cdcd admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:50.901694 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-3649e5b8-2d41-4d50-a722-b0285338cdcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:50.922383 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649e5b8-2d41-4d50-a722-b0285338cdcd admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:50.929378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74d58c1d4a1a45e09c588ac1633de287', 'request_id': 'req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:50.941716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74d58c1d4a1a45e09c588ac1633de287', 'request_id': 'req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:50.954929 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74d58c1d4a1a45e09c588ac1633de287', 'request_id': 'req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:50.996904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d admin admin] The token cache is being invalidate because project 74d58c1d4a1a45e09c588ac1633de287 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:23:50.997196 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:50.999509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:51.004208 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1262/2516] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:23:50 2022] DELETE /identity/v3/projects/74d58c1d4a1a45e09c588ac1633de287 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:51.009519 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:51.011194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:51.019416 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e0327-aff9-43b0-9581-a0dea4eb4870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:51.020050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e0327-aff9-43b0-9581-a0dea4eb4870 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.020434 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e0327-aff9-43b0-9581-a0dea4eb4870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.020676 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e0327-aff9-43b0-9581-a0dea4eb4870 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.021424 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e0327-aff9-43b0-9581-a0dea4eb4870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:51.023499 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e0327-aff9-43b0-9581-a0dea4eb4870 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:51.041815 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f865a3e-265f-4bf2-bba4-17fcdb5c900d admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:51.047579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d269471-6bfe-4921-867a-f31fafaf3d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:51.048268 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d269471-6bfe-4921-867a-f31fafaf3d54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.048542 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d269471-6bfe-4921-867a-f31fafaf3d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.048856 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d269471-6bfe-4921-867a-f31fafaf3d54 admin admin] PATH_INFO: `/v3/projects/ec88118aab0d4b1eadb56b15f25d3a85/users/b880a0eeb2cd4a6d8a3a212921017427/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.068603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d269471-6bfe-4921-867a-f31fafaf3d54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec88118aab0d4b1eadb56b15f25d3a85, user_id=b880a0eeb2cd4a6d8a3a212921017427, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:51.069956 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1255/2517] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:23:50 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:51.071491 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c93ce7-bcab-4c82-9df9-bd74c68caff7 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.071613 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d269471-6bfe-4921-867a-f31fafaf3d54 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:51.071810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c93ce7-bcab-4c82-9df9-bd74c68caff7 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.072024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c93ce7-bcab-4c82-9df9-bd74c68caff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.083772 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:51.095801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1263/2518] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:51 2022] PUT /identity/v3/projects/ec88118aab0d4b1eadb56b15f25d3a85/users/b880a0eeb2cd4a6d8a3a212921017427/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:51.097052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-90c93ce7-bcab-4c82-9df9-bd74c68caff7 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:51.097245 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bf482-55c9-44e0-8323-80fc4942f4dd None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.097535 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bf482-55c9-44e0-8323-80fc4942f4dd None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.097818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148bf482-55c9-44e0-8323-80fc4942f4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.098829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-90c93ce7-bcab-4c82-9df9-bd74c68caff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:51.119429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-148bf482-55c9-44e0-8323-80fc4942f4dd None None] MFA Rules not processed for user `e54e713b52e84393b018807f8153922e`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:51.120767 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-148bf482-55c9-44e0-8323-80fc4942f4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:51.131143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:51.180432 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1256/2519] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:51.187591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c93ce7-bcab-4c82-9df9-bd74c68caff7 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:51.187591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-90c93ce7-bcab-4c82-9df9-bd74c68caff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:51.192100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1264/2520] 10.222.0.45 () {58 vars in 1009 bytes} [Thu Aug 4 02:23:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:51.193005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bf482-55c9-44e0-8323-80fc4942f4dd None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:51.197619 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c93ce7-bcab-4c82-9df9-bd74c68caff7 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:51.209907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424fe6c4-807f-4a5f-8928-2b549ee01d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:51.210647 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fe6c4-807f-4a5f-8928-2b549ee01d2b admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.211098 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fe6c4-807f-4a5f-8928-2b549ee01d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.211378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424fe6c4-807f-4a5f-8928-2b549ee01d2b admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.212068 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424fe6c4-807f-4a5f-8928-2b549ee01d2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:51.215415 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424fe6c4-807f-4a5f-8928-2b549ee01d2b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:51.218114 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148bf482-55c9-44e0-8323-80fc4942f4dd None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:51.223383 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939d227-f3bd-41cc-9568-b2fe646534ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:51.224005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939d227-f3bd-41cc-9568-b2fe646534ec admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.224288 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939d227-f3bd-41cc-9568-b2fe646534ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.224566 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939d227-f3bd-41cc-9568-b2fe646534ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.225390 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939d227-f3bd-41cc-9568-b2fe646534ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:51.226999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939d227-f3bd-41cc-9568-b2fe646534ec admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:51.238950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1265/2521] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:51.243349 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1257/2522] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:23:51 2022] POST /identity/v3/users => generated 294 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:51.243567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5498b-6298-4259-a4fa-a7f600375d7c None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.243567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5498b-6298-4259-a4fa-a7f600375d7c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.243567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5498b-6298-4259-a4fa-a7f600375d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.243825 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424fe6c4-807f-4a5f-8928-2b549ee01d2b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:51.244964 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-424fe6c4-807f-4a5f-8928-2b549ee01d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:51.262743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-fbc5498b-6298-4259-a4fa-a7f600375d7c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:51.263944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-fbc5498b-6298-4259-a4fa-a7f600375d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:51.273316 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1266/2523] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:51.277419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc5498b-6298-4259-a4fa-a7f600375d7c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:51.287966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc5498b-6298-4259-a4fa-a7f600375d7c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:51.292995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a501c45-9c7e-4520-a448-64f8eb2a648b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:51.293683 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a501c45-9c7e-4520-a448-64f8eb2a648b admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.293962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a501c45-9c7e-4520-a448-64f8eb2a648b admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.294289 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a501c45-9c7e-4520-a448-64f8eb2a648b admin admin] PATH_INFO: `/v3/projects/ec88118aab0d4b1eadb56b15f25d3a85/users/b880a0eeb2cd4a6d8a3a212921017427/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.296326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a501c45-9c7e-4520-a448-64f8eb2a648b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec88118aab0d4b1eadb56b15f25d3a85, user_id=b880a0eeb2cd4a6d8a3a212921017427, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:51.299063 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a501c45-9c7e-4520-a448-64f8eb2a648b admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:51.322533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1267/2524] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:51 2022] PUT /identity/v3/projects/ec88118aab0d4b1eadb56b15f25d3a85/users/b880a0eeb2cd4a6d8a3a212921017427/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:51.322533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a501c45-9c7e-4520-a448-64f8eb2a648b admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:51.344430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424fe6c4-807f-4a5f-8928-2b549ee01d2b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:51.385410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a501c45-9c7e-4520-a448-64f8eb2a648b admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:51.390762 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb0dfa-aa03-4693-97ca-354b632e4c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:51.391382 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb0dfa-aa03-4693-97ca-354b632e4c70 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.391656 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb0dfa-aa03-4693-97ca-354b632e4c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.391962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb0dfa-aa03-4693-97ca-354b632e4c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.392676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb0dfa-aa03-4693-97ca-354b632e4c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:51.394259 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cb0dfa-aa03-4693-97ca-354b632e4c70 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:51.405220 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1268/2525] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:51.406079 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb0dfa-aa03-4693-97ca-354b632e4c70 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:51.412776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a4c6a-c585-49c0-9583-905f48b3a7a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '155a7276efab426cb34175683c475f33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:51.413411 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a4c6a-c585-49c0-9583-905f48b3a7a7 service swift] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.413694 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a4c6a-c585-49c0-9583-905f48b3a7a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.413966 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5a4c6a-c585-49c0-9583-905f48b3a7a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.415791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cb0dfa-aa03-4693-97ca-354b632e4c70 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:51.421341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d6ad8-784d-405c-a470-31e5933b9e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:51.422005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d6ad8-784d-405c-a470-31e5933b9e28 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.422300 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d6ad8-784d-405c-a470-31e5933b9e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.422583 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d6ad8-784d-405c-a470-31e5933b9e28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.423627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d6ad8-784d-405c-a470-31e5933b9e28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:23:51.423964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d6ad8-784d-405c-a470-31e5933b9e28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:51.426075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d6ad8-784d-405c-a470-31e5933b9e28 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:51.442678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1269/2526] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:23:51.442678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cb51f-3927-47ad-a3a8-b38cd76023ac None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.442678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cb51f-3927-47ad-a3a8-b38cd76023ac None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.442678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5cb51f-3927-47ad-a3a8-b38cd76023ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.449578 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:51.470874 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-9a5cb51f-3927-47ad-a3a8-b38cd76023ac None None] MFA Rules not processed for user `b880a0eeb2cd4a6d8a3a212921017427`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:51.472109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9a5cb51f-3927-47ad-a3a8-b38cd76023ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:51.478167 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a4c6a-c585-49c0-9583-905f48b3a7a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:51.479800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5a4c6a-c585-49c0-9583-905f48b3a7a7 service swift] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:51.487969 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1258/2527] 10.222.0.45 () {60 vars in 1384 bytes} [Thu Aug 4 02:23:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 02:23:51.488849 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a4c6a-c585-49c0-9583-905f48b3a7a7 service swift] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:51.498785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5a4c6a-c585-49c0-9583-905f48b3a7a7 service swift] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:51.504355 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bdbb7-3e68-48d0-b487-b1b48e3e68f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:51.505510 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bdbb7-3e68-48d0-b487-b1b48e3e68f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.506009 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bdbb7-3e68-48d0-b487-b1b48e3e68f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.506343 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bdbb7-3e68-48d0-b487-b1b48e3e68f6 admin admin] PATH_INFO: `/v3/projects/532331a91f4f4c7283dbbcbce8804160/users/9637ce56ef2b4e41a3d485ff565a1ee4/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.526153 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bdbb7-3e68-48d0-b487-b1b48e3e68f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=532331a91f4f4c7283dbbcbce8804160, user_id=9637ce56ef2b4e41a3d485ff565a1ee4, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:51.529081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bdbb7-3e68-48d0-b487-b1b48e3e68f6 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:51.547114 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:51.550389 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1259/2528] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:51 2022] PUT /identity/v3/projects/532331a91f4f4c7283dbbcbce8804160/users/9637ce56ef2b4e41a3d485ff565a1ee4/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:51.551296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bdbb7-3e68-48d0-b487-b1b48e3e68f6 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:51.563206 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1270/2529] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:51 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:51.564135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cb51f-3927-47ad-a3a8-b38cd76023ac None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:51.575019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5cb51f-3927-47ad-a3a8-b38cd76023ac None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:51.627142 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bdbb7-3e68-48d0-b487-b1b48e3e68f6 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:51.633292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bbb06-aa9a-49a5-b605-b0e3d4e46746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:51.633991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bbb06-aa9a-49a5-b605-b0e3d4e46746 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.634338 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bbb06-aa9a-49a5-b605-b0e3d4e46746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.634792 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2bbb06-aa9a-49a5-b605-b0e3d4e46746 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.635687 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2bbb06-aa9a-49a5-b605-b0e3d4e46746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:51.638404 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2bbb06-aa9a-49a5-b605-b0e3d4e46746 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:51.650175 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d902a0-907b-4e68-b6c8-d1795d9c5578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:51.650816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d902a0-907b-4e68-b6c8-d1795d9c5578 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.651098 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d902a0-907b-4e68-b6c8-d1795d9c5578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.651378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d902a0-907b-4e68-b6c8-d1795d9c5578 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.652202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d902a0-907b-4e68-b6c8-d1795d9c5578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:51.654051 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d902a0-907b-4e68-b6c8-d1795d9c5578 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:51.669048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1271/2530] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:51.669048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d902a0-907b-4e68-b6c8-d1795d9c5578 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:51.669582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-67d902a0-907b-4e68-b6c8-d1795d9c5578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:51.677401 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1260/2531] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:23:51 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:51.677401 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bbb06-aa9a-49a5-b605-b0e3d4e46746 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:51.743571 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d902a0-907b-4e68-b6c8-d1795d9c5578 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:51.760218 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2bbb06-aa9a-49a5-b605-b0e3d4e46746 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:51.766558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8803f-f3b9-4c83-a398-b81b76cb85dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:51.767254 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8803f-f3b9-4c83-a398-b81b76cb85dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.767570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8803f-f3b9-4c83-a398-b81b76cb85dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.767853 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f8803f-f3b9-4c83-a398-b81b76cb85dd admin admin] PATH_INFO: `/v3/projects/532331a91f4f4c7283dbbcbce8804160/users/9637ce56ef2b4e41a3d485ff565a1ee4/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.769992 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8803f-f3b9-4c83-a398-b81b76cb85dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=532331a91f4f4c7283dbbcbce8804160, user_id=9637ce56ef2b4e41a3d485ff565a1ee4, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:51.773933 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f8803f-f3b9-4c83-a398-b81b76cb85dd admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:51.792663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1261/2532] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:51 2022] PUT /identity/v3/projects/532331a91f4f4c7283dbbcbce8804160/users/9637ce56ef2b4e41a3d485ff565a1ee4/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:51.792663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8803f-f3b9-4c83-a398-b81b76cb85dd admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:51.814720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10049123-a4df-4a21-b558-0bd32c6900fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:51.815341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10049123-a4df-4a21-b558-0bd32c6900fd service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.818383 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10049123-a4df-4a21-b558-0bd32c6900fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.818383 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10049123-a4df-4a21-b558-0bd32c6900fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.843597 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:51.902099 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f8803f-f3b9-4c83-a398-b81b76cb85dd admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:51.908466 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec810f9a-05c0-43e5-a31a-64e619ce4420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:51.910509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec810f9a-05c0-43e5-a31a-64e619ce4420 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:51.911056 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec810f9a-05c0-43e5-a31a-64e619ce4420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:51.923125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec810f9a-05c0-43e5-a31a-64e619ce4420 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:51.923854 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10049123-a4df-4a21-b558-0bd32c6900fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:51.925877 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10049123-a4df-4a21-b558-0bd32c6900fd service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:51.929569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec810f9a-05c0-43e5-a31a-64e619ce4420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:51.931878 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec810f9a-05c0-43e5-a31a-64e619ce4420 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:51.936115 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:51.951772 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1272/2533] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:51 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:51.981812 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1262/2534] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:23:51 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:23:51.987917 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10049123-a4df-4a21-b558-0bd32c6900fd service cinder] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:52.004161 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10049123-a4df-4a21-b558-0bd32c6900fd service cinder] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:52.009796 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ccfc18-24b6-4802-86dc-2a835fee5506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:52.010603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccfc18-24b6-4802-86dc-2a835fee5506 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:52.011058 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccfc18-24b6-4802-86dc-2a835fee5506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:52.011502 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccfc18-24b6-4802-86dc-2a835fee5506 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:52.012417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ccfc18-24b6-4802-86dc-2a835fee5506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:52.014476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ccfc18-24b6-4802-86dc-2a835fee5506 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:52.032682 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1273/2535] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:23:52.034537 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec810f9a-05c0-43e5-a31a-64e619ce4420 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:52.051155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec810f9a-05c0-43e5-a31a-64e619ce4420 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:52.064914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190c2d9-ae9f-46e9-92f7-875494324bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:52.065839 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190c2d9-ae9f-46e9-92f7-875494324bdf admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:52.066153 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190c2d9-ae9f-46e9-92f7-875494324bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:52.066527 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190c2d9-ae9f-46e9-92f7-875494324bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:52.067783 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190c2d9-ae9f-46e9-92f7-875494324bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:52.069711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190c2d9-ae9f-46e9-92f7-875494324bdf admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:52.098328 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1263/2536] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:23:52.099798 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ccfc18-24b6-4802-86dc-2a835fee5506 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:52.118150 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ccfc18-24b6-4802-86dc-2a835fee5506 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:52.125609 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190c2d9-ae9f-46e9-92f7-875494324bdf admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:52.125864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b9f38-0d05-46a2-96f6-f00884a32aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:52.126439 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b9f38-0d05-46a2-96f6-f00884a32aed admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:52.126804 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b9f38-0d05-46a2-96f6-f00884a32aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:52.127125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b9f38-0d05-46a2-96f6-f00884a32aed admin admin] PATH_INFO: `/v3/projects/6e464a33c8244695b19722e2f27020fd/users/c8e0eef3e17f4f27b5048316a7a74dc2/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:52.148667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b9f38-0d05-46a2-96f6-f00884a32aed admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e464a33c8244695b19722e2f27020fd, user_id=c8e0eef3e17f4f27b5048316a7a74dc2, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:52.151848 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b9f38-0d05-46a2-96f6-f00884a32aed admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:52.177274 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1274/2537] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:23:52 2022] PUT /identity/v3/projects/6e464a33c8244695b19722e2f27020fd/users/c8e0eef3e17f4f27b5048316a7a74dc2/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:52.189794 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f65e5-0671-4f06-bf39-46c9df3ceee4 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:52.190190 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f65e5-0671-4f06-bf39-46c9df3ceee4 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:52.190469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f65e5-0671-4f06-bf39-46c9df3ceee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:52.235875 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c30f65e5-0671-4f06-bf39-46c9df3ceee4 None None] MFA Rules not processed for user `c8e0eef3e17f4f27b5048316a7a74dc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:52.237504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c30f65e5-0671-4f06-bf39-46c9df3ceee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:52.286547 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190c2d9-ae9f-46e9-92f7-875494324bdf admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:52.396982 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1275/2538] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:52.411013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a9d0f-4037-4bd6-b61b-a59a9b0907d6 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:52.411568 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a9d0f-4037-4bd6-b61b-a59a9b0907d6 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:52.412009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a9d0f-4037-4bd6-b61b-a59a9b0907d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:52.437598 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad142ebf-d326-4ceb-942b-eef84001e80c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:52.438247 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad142ebf-d326-4ceb-942b-eef84001e80c service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:52.438530 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad142ebf-d326-4ceb-942b-eef84001e80c service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:52.439011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad142ebf-d326-4ceb-942b-eef84001e80c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:52.440660 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ad142ebf-d326-4ceb-942b-eef84001e80c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:52.443028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-d89a9d0f-4037-4bd6-b61b-a59a9b0907d6 None None] MFA Rules not processed for user `e54e713b52e84393b018807f8153922e`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:52.445066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d89a9d0f-4037-4bd6-b61b-a59a9b0907d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:52.544413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad142ebf-d326-4ceb-942b-eef84001e80c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:52.545945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad142ebf-d326-4ceb-942b-eef84001e80c service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:52.558460 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1264/2539] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:52 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:52.559511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1276/2540] 10.222.0.45 () {58 vars in 967 bytes} [Thu Aug 4 02:23:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:52.565574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad142ebf-d326-4ceb-942b-eef84001e80c service cinder] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:52.567079 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ad142ebf-d326-4ceb-942b-eef84001e80c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:52.576290 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a9d0f-4037-4bd6-b61b-a59a9b0907d6 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:52.600749 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:52.702423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad142ebf-d326-4ceb-942b-eef84001e80c service cinder] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:52.713742 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d17d0f-904c-4fb2-b71d-d5dcc6884e84 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ff31d52ecf4c7d93a65bbfca7285c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:52.714389 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:52.714516 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d17d0f-904c-4fb2-b71d-d5dcc6884e84 service glance] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:52.714673 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d17d0f-904c-4fb2-b71d-d5dcc6884e84 service glance] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:52.714947 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d17d0f-904c-4fb2-b71d-d5dcc6884e84 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:52.732196 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a9d0f-4037-4bd6-b61b-a59a9b0907d6 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:52.737824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d17d0f-904c-4fb2-b71d-d5dcc6884e84 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:52.739466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8652ac-b78c-447c-a814-c7728b1c8765 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '155a7276efab426cb34175683c475f33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:52.740546 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8652ac-b78c-447c-a814-c7728b1c8765 service swift] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:52.740860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8652ac-b78c-447c-a814-c7728b1c8765 service swift] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:52.741275 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8652ac-b78c-447c-a814-c7728b1c8765 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:52.743679 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d17d0f-904c-4fb2-b71d-d5dcc6884e84 service glance] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:52.757697 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8652ac-b78c-447c-a814-c7728b1c8765 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:52.759808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8652ac-b78c-447c-a814-c7728b1c8765 service swift] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:52.769501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1265/2541] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:52.771213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1277/2542] 10.222.0.45 () {60 vars in 1384 bytes} [Thu Aug 4 02:23:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 02:23:53.152674 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cabcfe-982f-4ded-b496-414bf58445e5 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:53.153137 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cabcfe-982f-4ded-b496-414bf58445e5 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:53.153363 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cabcfe-982f-4ded-b496-414bf58445e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:53.204811 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-16cabcfe-982f-4ded-b496-414bf58445e5 None None] MFA Rules not processed for user `e54e713b52e84393b018807f8153922e`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:53.209352 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-16cabcfe-982f-4ded-b496-414bf58445e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:53.229487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1266/2543] 10.222.0.45 () {58 vars in 1009 bytes} [Thu Aug 4 02:23:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:53.242818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8652ac-b78c-447c-a814-c7728b1c8765 service swift] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:53.257802 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8652ac-b78c-447c-a814-c7728b1c8765 service swift] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:53.271920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51519f6-396b-4bd1-9519-a9c4672fbe1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '155a7276efab426cb34175683c475f33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:53.272560 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51519f6-396b-4bd1-9519-a9c4672fbe1e service swift] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:53.272903 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51519f6-396b-4bd1-9519-a9c4672fbe1e service swift] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:53.273150 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51519f6-396b-4bd1-9519-a9c4672fbe1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:53.294113 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51519f6-396b-4bd1-9519-a9c4672fbe1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:53.295267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51519f6-396b-4bd1-9519-a9c4672fbe1e service swift] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:53.316797 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1278/2544] 10.222.0.45 () {60 vars in 1384 bytes} [Thu Aug 4 02:23:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 02:23:53.737776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cabcfe-982f-4ded-b496-414bf58445e5 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:53.740268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-16cabcfe-982f-4ded-b496-414bf58445e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:53.880789 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cabcfe-982f-4ded-b496-414bf58445e5 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:53.891369 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282896bf-dd22-40d3-972e-50f76d8c062b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:53.892130 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282896bf-dd22-40d3-972e-50f76d8c062b service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:53.892421 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282896bf-dd22-40d3-972e-50f76d8c062b service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:53.892724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282896bf-dd22-40d3-972e-50f76d8c062b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:53.898652 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-282896bf-dd22-40d3-972e-50f76d8c062b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:54.032645 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bb4d5-f7aa-4158-8acf-f03accccc0c5 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:54.033588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bb4d5-f7aa-4158-8acf-f03accccc0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:54.034759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bb4d5-f7aa-4158-8acf-f03accccc0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:54.079456 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282896bf-dd22-40d3-972e-50f76d8c062b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:54.081108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282896bf-dd22-40d3-972e-50f76d8c062b service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:54.092203 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-8e4bb4d5-f7aa-4158-8acf-f03accccc0c5 None None] MFA Rules not processed for user `e54e713b52e84393b018807f8153922e`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:54.094487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1267/2545] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:54.118921 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8e4bb4d5-f7aa-4158-8acf-f03accccc0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:54.153779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1279/2546] 10.222.0.45 () {58 vars in 967 bytes} [Thu Aug 4 02:23:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:23:54.170352 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282896bf-dd22-40d3-972e-50f76d8c062b service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:54.190668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282896bf-dd22-40d3-972e-50f76d8c062b service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:54.196743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac165b6-6e48-4145-a3cc-6cad7d77932d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '155a7276efab426cb34175683c475f33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:54.197456 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac165b6-6e48-4145-a3cc-6cad7d77932d service swift] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:54.197836 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac165b6-6e48-4145-a3cc-6cad7d77932d service swift] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:54.198209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac165b6-6e48-4145-a3cc-6cad7d77932d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:54.209686 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac165b6-6e48-4145-a3cc-6cad7d77932d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:54.211881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac165b6-6e48-4145-a3cc-6cad7d77932d service swift] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:54.223131 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1268/2547] 10.222.0.45 () {60 vars in 1384 bytes} [Thu Aug 4 02:23:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 02:23:57.508580 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bb4d5-f7aa-4158-8acf-f03accccc0c5 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:57.510075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8e4bb4d5-f7aa-4158-8acf-f03accccc0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:57.528527 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bb4d5-f7aa-4158-8acf-f03accccc0c5 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:57.542755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51be82e-a92f-46d3-a433-c5be1cc16ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:57.543555 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51be82e-a92f-46d3-a433-c5be1cc16ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:57.544121 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51be82e-a92f-46d3-a433-c5be1cc16ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:57.544240 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51be82e-a92f-46d3-a433-c5be1cc16ae7 admin admin] PATH_INFO: `/v3/users/937131b7fd0048bf97cc2d0b5c3504f2` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:57.545955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51be82e-a92f-46d3-a433-c5be1cc16ae7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=937131b7fd0048bf97cc2d0b5c3504f2)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:57.549520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51be82e-a92f-46d3-a433-c5be1cc16ae7 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:57.656500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a51be82e-a92f-46d3-a433-c5be1cc16ae7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '937131b7fd0048bf97cc2d0b5c3504f2'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:57.670713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a51be82e-a92f-46d3-a433-c5be1cc16ae7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '937131b7fd0048bf97cc2d0b5c3504f2'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:57.671771 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a51be82e-a92f-46d3-a433-c5be1cc16ae7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '937131b7fd0048bf97cc2d0b5c3504f2'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:57.705412 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a51be82e-a92f-46d3-a433-c5be1cc16ae7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '937131b7fd0048bf97cc2d0b5c3504f2'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:57.706156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a51be82e-a92f-46d3-a433-c5be1cc16ae7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '937131b7fd0048bf97cc2d0b5c3504f2'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:57.737500 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1280/2548] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:23:57 2022] DELETE /identity/v3/users/937131b7fd0048bf97cc2d0b5c3504f2 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:57.737746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ea8a5-5572-449a-b140-449493051542 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:57.737746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ea8a5-5572-449a-b140-449493051542 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:57.737746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ea8a5-5572-449a-b140-449493051542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:57.782524 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-930ea8a5-5572-449a-b140-449493051542 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:57.783691 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-930ea8a5-5572-449a-b140-449493051542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:57.895833 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:57.951216 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1269/2549] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:23:57.996751 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51be82e-a92f-46d3-a433-c5be1cc16ae7 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:57.998446 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a51be82e-a92f-46d3-a433-c5be1cc16ae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:58.149823 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ea8a5-5572-449a-b140-449493051542 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:58.152841 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-930ea8a5-5572-449a-b140-449493051542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:58.179786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ea8a5-5572-449a-b140-449493051542 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:58.181858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51be82e-a92f-46d3-a433-c5be1cc16ae7 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:58.190862 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298bab96-cbac-4e92-b6d5-96a746b98e69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:58.191744 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bab96-cbac-4e92-b6d5-96a746b98e69 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:58.192056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bab96-cbac-4e92-b6d5-96a746b98e69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:58.192340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298bab96-cbac-4e92-b6d5-96a746b98e69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:58.212367 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acada16-4d45-48fb-ad38-6995e493af2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:58.213952 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acada16-4d45-48fb-ad38-6995e493af2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:58.214519 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acada16-4d45-48fb-ad38-6995e493af2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:58.214712 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acada16-4d45-48fb-ad38-6995e493af2a admin admin] PATH_INFO: `/v3/users/9d9dbc4b93504f9eab5fb7425d9ce3f1` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:58.216416 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acada16-4d45-48fb-ad38-6995e493af2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d9dbc4b93504f9eab5fb7425d9ce3f1)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:58.220768 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acada16-4d45-48fb-ad38-6995e493af2a admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:58.226950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bab96-cbac-4e92-b6d5-96a746b98e69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:58.228416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298bab96-cbac-4e92-b6d5-96a746b98e69 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:58.252677 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1281/2550] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:58.348732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8acada16-4d45-48fb-ad38-6995e493af2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d9dbc4b93504f9eab5fb7425d9ce3f1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:58.349666 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8acada16-4d45-48fb-ad38-6995e493af2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d9dbc4b93504f9eab5fb7425d9ce3f1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:58.350732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8acada16-4d45-48fb-ad38-6995e493af2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d9dbc4b93504f9eab5fb7425d9ce3f1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:58.373168 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8acada16-4d45-48fb-ad38-6995e493af2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d9dbc4b93504f9eab5fb7425d9ce3f1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:58.385618 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-8acada16-4d45-48fb-ad38-6995e493af2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d9dbc4b93504f9eab5fb7425d9ce3f1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:58.415173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1270/2551] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:23:58 2022] DELETE /identity/v3/users/9d9dbc4b93504f9eab5fb7425d9ce3f1 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:58.416149 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec428d-9e67-4821-a77d-c32180fc1073 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:58.416692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec428d-9e67-4821-a77d-c32180fc1073 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:58.417348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec428d-9e67-4821-a77d-c32180fc1073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:58.469159 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c2ec428d-9e67-4821-a77d-c32180fc1073 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:58.472819 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c2ec428d-9e67-4821-a77d-c32180fc1073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:58.570004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:58.610172 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1282/2552] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:23:58.626591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acada16-4d45-48fb-ad38-6995e493af2a admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:58.628192 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-8acada16-4d45-48fb-ad38-6995e493af2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:58.744047 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acada16-4d45-48fb-ad38-6995e493af2a admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:58.756926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee851377-94fb-4edc-9055-921b1276539e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:58.757960 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee851377-94fb-4edc-9055-921b1276539e service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:58.758466 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee851377-94fb-4edc-9055-921b1276539e service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:58.758986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee851377-94fb-4edc-9055-921b1276539e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:58.775625 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee851377-94fb-4edc-9055-921b1276539e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:58.777508 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee851377-94fb-4edc-9055-921b1276539e service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:58.802520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1271/2553] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:23:59.024164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec428d-9e67-4821-a77d-c32180fc1073 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:59.025624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c2ec428d-9e67-4821-a77d-c32180fc1073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:59.046909 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec428d-9e67-4821-a77d-c32180fc1073 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:59.061315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28826e95-48ab-4e04-a7b7-fcba0a192be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:59.061315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28826e95-48ab-4e04-a7b7-fcba0a192be8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:59.061315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28826e95-48ab-4e04-a7b7-fcba0a192be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:59.061315 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28826e95-48ab-4e04-a7b7-fcba0a192be8 admin admin] PATH_INFO: `/v3/projects/19eead1456de4c6b9a2a0b0524674f0f` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:59.065184 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28826e95-48ab-4e04-a7b7-fcba0a192be8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19eead1456de4c6b9a2a0b0524674f0f)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:59.067360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28826e95-48ab-4e04-a7b7-fcba0a192be8 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:59.205397 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-28826e95-48ab-4e04-a7b7-fcba0a192be8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19eead1456de4c6b9a2a0b0524674f0f', 'request_id': 'req-28826e95-48ab-4e04-a7b7-fcba0a192be8'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:59.218621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-28826e95-48ab-4e04-a7b7-fcba0a192be8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19eead1456de4c6b9a2a0b0524674f0f', 'request_id': 'req-28826e95-48ab-4e04-a7b7-fcba0a192be8'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:59.247830 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-28826e95-48ab-4e04-a7b7-fcba0a192be8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19eead1456de4c6b9a2a0b0524674f0f', 'request_id': 'req-28826e95-48ab-4e04-a7b7-fcba0a192be8'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:59.303508 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-28826e95-48ab-4e04-a7b7-fcba0a192be8 admin admin] The token cache is being invalidate because project 19eead1456de4c6b9a2a0b0524674f0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:23:59.303816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-28826e95-48ab-4e04-a7b7-fcba0a192be8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:59.304670 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-28826e95-48ab-4e04-a7b7-fcba0a192be8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:59.313433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bab127-4197-48e5-8202-10e960ba932a None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:59.313786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bab127-4197-48e5-8202-10e960ba932a None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:59.314142 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bab127-4197-48e5-8202-10e960ba932a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:59.316351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1283/2554] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:23:59 2022] DELETE /identity/v3/projects/19eead1456de4c6b9a2a0b0524674f0f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:59.376791 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d1927-51e5-43f0-85d8-10d7db17dcab None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:59.377195 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d1927-51e5-43f0-85d8-10d7db17dcab None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:59.377595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d1927-51e5-43f0-85d8-10d7db17dcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:59.377765 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-21bab127-4197-48e5-8202-10e960ba932a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:59.383116 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-21bab127-4197-48e5-8202-10e960ba932a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:59.434896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-2f4d1927-51e5-43f0-85d8-10d7db17dcab None None] MFA Rules not processed for user `9637ce56ef2b4e41a3d485ff565a1ee4`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:59.437566 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-2f4d1927-51e5-43f0-85d8-10d7db17dcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:59.535054 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1272/2555] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:59.548090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1c0a8-1722-46f1-9dbc-9c33358805c7 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:59.549291 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1c0a8-1722-46f1-9dbc-9c33358805c7 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:59.549733 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e1c0a8-1722-46f1-9dbc-9c33358805c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:59.567489 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:59.587427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-d8e1c0a8-1722-46f1-9dbc-9c33358805c7 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:23:59.589961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d8e1c0a8-1722-46f1-9dbc-9c33358805c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:59.614371 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1273/2556] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:23:59.616347 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1c0a8-1722-46f1-9dbc-9c33358805c7 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:59.618141 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d8e1c0a8-1722-46f1-9dbc-9c33358805c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:59.619358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1284/2557] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:23:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:23:59.620859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d1927-51e5-43f0-85d8-10d7db17dcab None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:59.634921 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e1c0a8-1722-46f1-9dbc-9c33358805c7 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:59.641902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14926573-83af-4a58-9843-c86a14014d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:59.642799 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14926573-83af-4a58-9843-c86a14014d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:59.643414 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14926573-83af-4a58-9843-c86a14014d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:59.644178 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14926573-83af-4a58-9843-c86a14014d5e admin admin] PATH_INFO: `/v3/projects/bff29fe8fd7f4a8e9d05a0f8e4ab839d` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:59.646458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14926573-83af-4a58-9843-c86a14014d5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bff29fe8fd7f4a8e9d05a0f8e4ab839d)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:59.649470 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14926573-83af-4a58-9843-c86a14014d5e admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:59.659429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d1927-51e5-43f0-85d8-10d7db17dcab None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:59.669289 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300aed6-3b9d-40fb-b5b7-f3d2035bb54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:23:59.671125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300aed6-3b9d-40fb-b5b7-f3d2035bb54a admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:23:59.671920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300aed6-3b9d-40fb-b5b7-f3d2035bb54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:23:59.672472 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a300aed6-3b9d-40fb-b5b7-f3d2035bb54a admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:23:59.673798 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300aed6-3b9d-40fb-b5b7-f3d2035bb54a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:23:59.674110 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300aed6-3b9d-40fb-b5b7-f3d2035bb54a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:23:59.677525 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a300aed6-3b9d-40fb-b5b7-f3d2035bb54a admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:23:59.708577 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1285/2558] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:23:59.713567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300aed6-3b9d-40fb-b5b7-f3d2035bb54a admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:59.717537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a300aed6-3b9d-40fb-b5b7-f3d2035bb54a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:59.750138 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-14926573-83af-4a58-9843-c86a14014d5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bff29fe8fd7f4a8e9d05a0f8e4ab839d', 'request_id': 'req-14926573-83af-4a58-9843-c86a14014d5e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:59.769417 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-14926573-83af-4a58-9843-c86a14014d5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bff29fe8fd7f4a8e9d05a0f8e4ab839d', 'request_id': 'req-14926573-83af-4a58-9843-c86a14014d5e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:59.777450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-14926573-83af-4a58-9843-c86a14014d5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bff29fe8fd7f4a8e9d05a0f8e4ab839d', 'request_id': 'req-14926573-83af-4a58-9843-c86a14014d5e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:59.801681 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-14926573-83af-4a58-9843-c86a14014d5e admin admin] The token cache is being invalidate because project bff29fe8fd7f4a8e9d05a0f8e4ab839d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:23:59.801982 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-14926573-83af-4a58-9843-c86a14014d5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:59.803698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-14926573-83af-4a58-9843-c86a14014d5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:23:59.807343 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1274/2559] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:23:59 2022] DELETE /identity/v3/projects/bff29fe8fd7f4a8e9d05a0f8e4ab839d => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:23:59.807720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14926573-83af-4a58-9843-c86a14014d5e admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:23:59.816883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-14926573-83af-4a58-9843-c86a14014d5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:23:59.824689 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:23:59.899449 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a300aed6-3b9d-40fb-b5b7-f3d2035bb54a admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:23:59.999899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14926573-83af-4a58-9843-c86a14014d5e admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:00.006243 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e2d94-f785-4a20-af36-74134ef9fc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:00.007928 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e2d94-f785-4a20-af36-74134ef9fc2c admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:00.008418 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e2d94-f785-4a20-af36-74134ef9fc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:00.009164 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e2d94-f785-4a20-af36-74134ef9fc2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:00.010039 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e2d94-f785-4a20-af36-74134ef9fc2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:00.012366 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e2d94-f785-4a20-af36-74134ef9fc2c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:00.025151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ae4f0-6cc5-457f-8278-39e9dde4067e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:00.025953 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ae4f0-6cc5-457f-8278-39e9dde4067e service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:00.026333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ae4f0-6cc5-457f-8278-39e9dde4067e service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:00.026706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ae4f0-6cc5-457f-8278-39e9dde4067e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:00.034147 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-097ae4f0-6cc5-457f-8278-39e9dde4067e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:00.093681 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1275/2560] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:23:59 2022] POST /identity/v3/projects => generated 385 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:00.095303 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e2d94-f785-4a20-af36-74134ef9fc2c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:00.103878 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-715e2d94-f785-4a20-af36-74134ef9fc2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:00.153127 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ae4f0-6cc5-457f-8278-39e9dde4067e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:00.156940 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ae4f0-6cc5-457f-8278-39e9dde4067e service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:00.169455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1286/2561] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:23:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:00.178629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ae4f0-6cc5-457f-8278-39e9dde4067e service nova] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:00.222151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ae4f0-6cc5-457f-8278-39e9dde4067e service nova] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:00.313154 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e2d94-f785-4a20-af36-74134ef9fc2c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:00.324963 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5d4eb-4459-499a-8c4d-511253e22cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:00.325702 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5d4eb-4459-499a-8c4d-511253e22cb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:00.326059 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5d4eb-4459-499a-8c4d-511253e22cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:00.326410 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5d4eb-4459-499a-8c4d-511253e22cb1 admin admin] PATH_INFO: `/v3/users/b880a0eeb2cd4a6d8a3a212921017427` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:00.327980 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5d4eb-4459-499a-8c4d-511253e22cb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b880a0eeb2cd4a6d8a3a212921017427)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:00.330302 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5d4eb-4459-499a-8c4d-511253e22cb1 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:00.344675 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:00.383618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c83280-ed8a-4819-b340-405f03c4c722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:00.384487 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c83280-ed8a-4819-b340-405f03c4c722 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:00.384868 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c83280-ed8a-4819-b340-405f03c4c722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:00.385886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c83280-ed8a-4819-b340-405f03c4c722 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:00.385886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c83280-ed8a-4819-b340-405f03c4c722 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:00.388057 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c83280-ed8a-4819-b340-405f03c4c722 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:00.428243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1287/2562] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:24:00 2022] POST /identity/v3/users => generated 293 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:00.429096 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-5dd5d4eb-4459-499a-8c4d-511253e22cb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b880a0eeb2cd4a6d8a3a212921017427'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:00.429829 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c83280-ed8a-4819-b340-405f03c4c722 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:00.430128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-5dd5d4eb-4459-499a-8c4d-511253e22cb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b880a0eeb2cd4a6d8a3a212921017427'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:00.430998 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-5dd5d4eb-4459-499a-8c4d-511253e22cb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b880a0eeb2cd4a6d8a3a212921017427'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:00.433463 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-58c83280-ed8a-4819-b340-405f03c4c722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:00.442825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:00.449923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-5dd5d4eb-4459-499a-8c4d-511253e22cb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b880a0eeb2cd4a6d8a3a212921017427'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:00.453476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c83280-ed8a-4819-b340-405f03c4c722 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:00.459634 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e5548-e65d-418a-9364-5e8e9e408e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:00.460488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-5dd5d4eb-4459-499a-8c4d-511253e22cb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b880a0eeb2cd4a6d8a3a212921017427'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:00.460703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e5548-e65d-418a-9364-5e8e9e408e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:00.461089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e5548-e65d-418a-9364-5e8e9e408e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:00.461843 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09e5548-e65d-418a-9364-5e8e9e408e6c admin admin] PATH_INFO: `/v3/users/df5d3de1afa945c3b0873050eca06c0c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:00.463089 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e5548-e65d-418a-9364-5e8e9e408e6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=df5d3de1afa945c3b0873050eca06c0c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:00.465198 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09e5548-e65d-418a-9364-5e8e9e408e6c admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:00.487410 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1276/2563] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:24:00 2022] DELETE /identity/v3/users/b880a0eeb2cd4a6d8a3a212921017427 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:00.488545 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5d4eb-4459-499a-8c4d-511253e22cb1 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:00.490296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-5dd5d4eb-4459-499a-8c4d-511253e22cb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:00.543099 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a09e5548-e65d-418a-9364-5e8e9e408e6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df5d3de1afa945c3b0873050eca06c0c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:00.561696 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a09e5548-e65d-418a-9364-5e8e9e408e6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df5d3de1afa945c3b0873050eca06c0c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:00.563889 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a09e5548-e65d-418a-9364-5e8e9e408e6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df5d3de1afa945c3b0873050eca06c0c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:00.581536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a09e5548-e65d-418a-9364-5e8e9e408e6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df5d3de1afa945c3b0873050eca06c0c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:00.582915 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a09e5548-e65d-418a-9364-5e8e9e408e6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df5d3de1afa945c3b0873050eca06c0c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:00.613047 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1288/2564] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:24:00 2022] DELETE /identity/v3/users/df5d3de1afa945c3b0873050eca06c0c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:00.632746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5d4eb-4459-499a-8c4d-511253e22cb1 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:00.752000 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83234a76-f692-4d1f-9421-c86334ddef94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:00.752906 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83234a76-f692-4d1f-9421-c86334ddef94 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:00.753286 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83234a76-f692-4d1f-9421-c86334ddef94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:00.753682 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83234a76-f692-4d1f-9421-c86334ddef94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:00.754500 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83234a76-f692-4d1f-9421-c86334ddef94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:00.756766 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83234a76-f692-4d1f-9421-c86334ddef94 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:00.772902 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1277/2565] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:00.778073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e5548-e65d-418a-9364-5e8e9e408e6c admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:00.779531 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a09e5548-e65d-418a-9364-5e8e9e408e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:00.802519 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09e5548-e65d-418a-9364-5e8e9e408e6c admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:00.810355 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef32914-7359-4594-a363-8ad2a1b49a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:00.811147 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef32914-7359-4594-a363-8ad2a1b49a1f admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:00.811679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef32914-7359-4594-a363-8ad2a1b49a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:00.811847 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef32914-7359-4594-a363-8ad2a1b49a1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:00.811847 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef32914-7359-4594-a363-8ad2a1b49a1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:00.815130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef32914-7359-4594-a363-8ad2a1b49a1f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:00.831600 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1289/2566] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:00 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:00.841005 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83234a76-f692-4d1f-9421-c86334ddef94 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:00.853301 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83234a76-f692-4d1f-9421-c86334ddef94 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:00.861139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8397f-a254-4d21-aa1c-d5b1ace73e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:00.861988 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8397f-a254-4d21-aa1c-d5b1ace73e12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:00.862612 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8397f-a254-4d21-aa1c-d5b1ace73e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:00.862612 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d8397f-a254-4d21-aa1c-d5b1ace73e12 admin admin] PATH_INFO: `/v3/projects/06f7c91a924c447fbca1b6a784d7b167/users/fb3a977fea2c499eb915bc059ab01a79/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:00.900970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8397f-a254-4d21-aa1c-d5b1ace73e12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06f7c91a924c447fbca1b6a784d7b167, user_id=fb3a977fea2c499eb915bc059ab01a79, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:00.904380 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d8397f-a254-4d21-aa1c-d5b1ace73e12 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:00.923453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1278/2567] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:00 2022] PUT /identity/v3/projects/06f7c91a924c447fbca1b6a784d7b167/users/fb3a977fea2c499eb915bc059ab01a79/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:00.929570 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68974642-318e-4dcf-ba27-365b29d288da None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:00.929860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68974642-318e-4dcf-ba27-365b29d288da None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:00.930169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68974642-318e-4dcf-ba27-365b29d288da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:00.963842 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8397f-a254-4d21-aa1c-d5b1ace73e12 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:00.966915 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-50d8397f-a254-4d21-aa1c-d5b1ace73e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:00.974302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-68974642-318e-4dcf-ba27-365b29d288da None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:00.976157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-68974642-318e-4dcf-ba27-365b29d288da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:00.982385 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:01.082398 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1290/2568] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:01.083336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68974642-318e-4dcf-ba27-365b29d288da None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:01.084907 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-68974642-318e-4dcf-ba27-365b29d288da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:01.101611 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68974642-318e-4dcf-ba27-365b29d288da None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:01.106859 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d8397f-a254-4d21-aa1c-d5b1ace73e12 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:01.109772 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5340f4-1ca3-425b-95bc-318708f43941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:01.110602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5340f4-1ca3-425b-95bc-318708f43941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.110886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5340f4-1ca3-425b-95bc-318708f43941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.111167 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5340f4-1ca3-425b-95bc-318708f43941 admin admin] PATH_INFO: `/v3/projects/ec88118aab0d4b1eadb56b15f25d3a85` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.112404 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5340f4-1ca3-425b-95bc-318708f43941 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec88118aab0d4b1eadb56b15f25d3a85)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:01.113007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a74e0a-4d17-4416-a19c-90283726b868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:01.113732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a74e0a-4d17-4416-a19c-90283726b868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.114387 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a74e0a-4d17-4416-a19c-90283726b868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.114508 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5340f4-1ca3-425b-95bc-318708f43941 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:01.114615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a74e0a-4d17-4416-a19c-90283726b868 admin admin] PATH_INFO: `/v3/projects/11284ae13d2f4c5ba47517f672968623` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.115758 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a74e0a-4d17-4416-a19c-90283726b868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11284ae13d2f4c5ba47517f672968623)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:01.117777 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a74e0a-4d17-4416-a19c-90283726b868 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:01.188305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3c5340f4-1ca3-425b-95bc-318708f43941 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec88118aab0d4b1eadb56b15f25d3a85', 'request_id': 'req-3c5340f4-1ca3-425b-95bc-318708f43941'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:01.193197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-93a74e0a-4d17-4416-a19c-90283726b868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11284ae13d2f4c5ba47517f672968623', 'request_id': 'req-93a74e0a-4d17-4416-a19c-90283726b868'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:01.203567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3c5340f4-1ca3-425b-95bc-318708f43941 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec88118aab0d4b1eadb56b15f25d3a85', 'request_id': 'req-3c5340f4-1ca3-425b-95bc-318708f43941'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:01.207238 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:01.212265 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-93a74e0a-4d17-4416-a19c-90283726b868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11284ae13d2f4c5ba47517f672968623', 'request_id': 'req-93a74e0a-4d17-4416-a19c-90283726b868'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:01.214795 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3c5340f4-1ca3-425b-95bc-318708f43941 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec88118aab0d4b1eadb56b15f25d3a85', 'request_id': 'req-3c5340f4-1ca3-425b-95bc-318708f43941'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:01.228853 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-93a74e0a-4d17-4416-a19c-90283726b868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11284ae13d2f4c5ba47517f672968623', 'request_id': 'req-93a74e0a-4d17-4416-a19c-90283726b868'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:01.232221 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3c5340f4-1ca3-425b-95bc-318708f43941 admin admin] The token cache is being invalidate because project ec88118aab0d4b1eadb56b15f25d3a85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:24:01.232939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3c5340f4-1ca3-425b-95bc-318708f43941 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:01.233630 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3c5340f4-1ca3-425b-95bc-318708f43941 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:01.235731 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1291/2569] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:24:01 2022] DELETE /identity/v3/projects/ec88118aab0d4b1eadb56b15f25d3a85 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:01.236590 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5340f4-1ca3-425b-95bc-318708f43941 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:01.238047 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-3c5340f4-1ca3-425b-95bc-318708f43941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:01.247337 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-93a74e0a-4d17-4416-a19c-90283726b868 admin admin] The token cache is being invalidate because project 11284ae13d2f4c5ba47517f672968623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:24:01.247631 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-93a74e0a-4d17-4416-a19c-90283726b868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:01.248324 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-93a74e0a-4d17-4416-a19c-90283726b868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:01.251661 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1279/2570] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:24:00 2022] DELETE /identity/v3/projects/11284ae13d2f4c5ba47517f672968623 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:01.253855 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff7269-6c18-4474-b933-86763addef89 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.254004 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff7269-6c18-4474-b933-86763addef89 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.254201 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff7269-6c18-4474-b933-86763addef89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.279074 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-7fff7269-6c18-4474-b933-86763addef89 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:01.280372 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7fff7269-6c18-4474-b933-86763addef89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:01.335760 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5340f4-1ca3-425b-95bc-318708f43941 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:01.348718 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fb0832-a66e-4ab3-8ce4-9c1a298ce421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:01.350049 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb0832-a66e-4ab3-8ce4-9c1a298ce421 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.350604 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb0832-a66e-4ab3-8ce4-9c1a298ce421 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.350893 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb0832-a66e-4ab3-8ce4-9c1a298ce421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.353019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-75fb0832-a66e-4ab3-8ce4-9c1a298ce421 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:01.359881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1280/2571] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:01.361355 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e0413-cd19-477b-903e-c6521189f0e7 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.361704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e0413-cd19-477b-903e-c6521189f0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.362134 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e0413-cd19-477b-903e-c6521189f0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.379246 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb0832-a66e-4ab3-8ce4-9c1a298ce421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:01.380824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fb0832-a66e-4ab3-8ce4-9c1a298ce421 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:01.391892 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1292/2572] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:01.393310 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfef484-8c2b-45a4-b4b4-933970a29cea None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.393658 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfef484-8c2b-45a4-b4b4-933970a29cea None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.393999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfef484-8c2b-45a4-b4b4-933970a29cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.402212 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-3f6e0413-cd19-477b-903e-c6521189f0e7 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:01.403441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-3f6e0413-cd19-477b-903e-c6521189f0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:01.419618 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1281/2573] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:01.421187 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97607da6-4ea2-4ae1-9918-3526e0726d3f None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.422113 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97607da6-4ea2-4ae1-9918-3526e0726d3f None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.422452 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97607da6-4ea2-4ae1-9918-3526e0726d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.423986 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-fbfef484-8c2b-45a4-b4b4-933970a29cea None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:01.427756 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-fbfef484-8c2b-45a4-b4b4-933970a29cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:01.443891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1293/2574] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:01.449696 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfef484-8c2b-45a4-b4b4-933970a29cea None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:01.453613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-97607da6-4ea2-4ae1-9918-3526e0726d3f None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:01.455816 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-97607da6-4ea2-4ae1-9918-3526e0726d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:01.461860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfef484-8c2b-45a4-b4b4-933970a29cea None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:01.467075 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52fc9f-7536-43fb-9e4d-4f6af8235b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:01.467509 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52fc9f-7536-43fb-9e4d-4f6af8235b7a admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.468165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52fc9f-7536-43fb-9e4d-4f6af8235b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.468613 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52fc9f-7536-43fb-9e4d-4f6af8235b7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.469906 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52fc9f-7536-43fb-9e4d-4f6af8235b7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:24:01.470319 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52fc9f-7536-43fb-9e4d-4f6af8235b7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:01.472197 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52fc9f-7536-43fb-9e4d-4f6af8235b7a admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:01.490841 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1282/2575] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:01.490841 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97607da6-4ea2-4ae1-9918-3526e0726d3f None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:01.490841 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97607da6-4ea2-4ae1-9918-3526e0726d3f None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:01.499008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1294/2576] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:24:01.504574 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52fc9f-7536-43fb-9e4d-4f6af8235b7a admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:01.505375 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e490b39-6f81-48cb-96a7-99beb1388bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:01.506426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e490b39-6f81-48cb-96a7-99beb1388bef admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.506701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e490b39-6f81-48cb-96a7-99beb1388bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.507053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e490b39-6f81-48cb-96a7-99beb1388bef admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.508101 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e490b39-6f81-48cb-96a7-99beb1388bef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:24:01.508505 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e490b39-6f81-48cb-96a7-99beb1388bef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:01.510354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e490b39-6f81-48cb-96a7-99beb1388bef admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:01.524330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52fc9f-7536-43fb-9e4d-4f6af8235b7a admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:01.531584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1283/2577] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:24:01.534012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97e8f5-8bb4-46fa-9f7e-ddec1e9541f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:01.535024 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97e8f5-8bb4-46fa-9f7e-ddec1e9541f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.536040 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e490b39-6f81-48cb-96a7-99beb1388bef admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:01.536789 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97e8f5-8bb4-46fa-9f7e-ddec1e9541f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.537563 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97e8f5-8bb4-46fa-9f7e-ddec1e9541f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.541417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97e8f5-8bb4-46fa-9f7e-ddec1e9541f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:01.544256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97e8f5-8bb4-46fa-9f7e-ddec1e9541f6 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:01.553335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e490b39-6f81-48cb-96a7-99beb1388bef admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:01.558450 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24214194-2b4b-448f-b1ea-baae708e99f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:01.559688 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24214194-2b4b-448f-b1ea-baae708e99f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.560055 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24214194-2b4b-448f-b1ea-baae708e99f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.560406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24214194-2b4b-448f-b1ea-baae708e99f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.561377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24214194-2b4b-448f-b1ea-baae708e99f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:01.563468 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24214194-2b4b-448f-b1ea-baae708e99f8 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:01.570611 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:01.598709 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1295/2578] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:24:01 2022] POST /identity/v3/projects => generated 389 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:01.599849 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97e8f5-8bb4-46fa-9f7e-ddec1e9541f6 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:01.614115 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1284/2579] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:24:01 2022] POST /identity/v3/projects => generated 371 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:01.618024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24214194-2b4b-448f-b1ea-baae708e99f8 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:01.700384 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97e8f5-8bb4-46fa-9f7e-ddec1e9541f6 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:01.757303 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24214194-2b4b-448f-b1ea-baae708e99f8 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:01.776581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ec2b3-94a5-4570-9175-a1b8a2b54d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:01.777594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ec2b3-94a5-4570-9175-a1b8a2b54d4d admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.778039 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ec2b3-94a5-4570-9175-a1b8a2b54d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.778515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ec2b3-94a5-4570-9175-a1b8a2b54d4d admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.779544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ec2b3-94a5-4570-9175-a1b8a2b54d4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:01.783376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ec2b3-94a5-4570-9175-a1b8a2b54d4d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:01.787610 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:01.799587 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cf055-d39c-406f-b316-5a56ed1be0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:01.800415 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cf055-d39c-406f-b316-5a56ed1be0cd admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.800767 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cf055-d39c-406f-b316-5a56ed1be0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.801370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cf055-d39c-406f-b316-5a56ed1be0cd admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.802321 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6cf055-d39c-406f-b316-5a56ed1be0cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:01.805123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6cf055-d39c-406f-b316-5a56ed1be0cd admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:01.815968 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1285/2580] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:24:01 2022] POST /identity/v3/users => generated 286 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:24:01.827667 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ec2b3-94a5-4570-9175-a1b8a2b54d4d admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:01.838383 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1296/2581] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:24:01 2022] POST /identity/v3/users => generated 295 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:24:01.839642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ec2b3-94a5-4570-9175-a1b8a2b54d4d admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:01.843736 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cf055-d39c-406f-b316-5a56ed1be0cd admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:01.846728 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf28fc2-1384-472a-897f-d3d6addda0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:01.847501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf28fc2-1384-472a-897f-d3d6addda0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.847889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf28fc2-1384-472a-897f-d3d6addda0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.848268 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf28fc2-1384-472a-897f-d3d6addda0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.849746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf28fc2-1384-472a-897f-d3d6addda0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:01.852606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf28fc2-1384-472a-897f-d3d6addda0b4 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:01.860865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cf055-d39c-406f-b316-5a56ed1be0cd admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:01.869026 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405f7b5-15c4-45a7-953f-c4d76f3490ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:01.869675 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405f7b5-15c4-45a7-953f-c4d76f3490ec admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.869960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405f7b5-15c4-45a7-953f-c4d76f3490ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.870354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2405f7b5-15c4-45a7-953f-c4d76f3490ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.878082 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1286/2582] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:01.878256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2405f7b5-15c4-45a7-953f-c4d76f3490ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:01.879398 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf28fc2-1384-472a-897f-d3d6addda0b4 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:01.881702 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2405f7b5-15c4-45a7-953f-c4d76f3490ec admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:01.892183 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf28fc2-1384-472a-897f-d3d6addda0b4 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:01.898150 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1297/2583] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:01.900784 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0433a-2902-4b70-9aca-543999ab7abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:01.901664 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0433a-2902-4b70-9aca-543999ab7abd admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.902052 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0433a-2902-4b70-9aca-543999ab7abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.902413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0433a-2902-4b70-9aca-543999ab7abd admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.903365 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0433a-2902-4b70-9aca-543999ab7abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:01.904903 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405f7b5-15c4-45a7-953f-c4d76f3490ec admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:01.906225 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0433a-2902-4b70-9aca-543999ab7abd admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:01.919262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1287/2584] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:01.923698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0433a-2902-4b70-9aca-543999ab7abd admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:01.928744 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2405f7b5-15c4-45a7-953f-c4d76f3490ec admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:01.936411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d288f-86d0-4ad9-9c82-e0eed09e2b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:01.937133 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d288f-86d0-4ad9-9c82-e0eed09e2b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.937419 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d288f-86d0-4ad9-9c82-e0eed09e2b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.937690 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d288f-86d0-4ad9-9c82-e0eed09e2b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.938452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d288f-86d0-4ad9-9c82-e0eed09e2b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:01.939031 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0433a-2902-4b70-9aca-543999ab7abd admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:01.940066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d288f-86d0-4ad9-9c82-e0eed09e2b12 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:01.947336 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8219cef1-481c-4f3c-9a49-412276149fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:01.948323 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8219cef1-481c-4f3c-9a49-412276149fcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.948829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8219cef1-481c-4f3c-9a49-412276149fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.953242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8219cef1-481c-4f3c-9a49-412276149fcf admin admin] PATH_INFO: `/v3/projects/069304da63a0488880fb09c6c3af2ce6/users/89877030552f4954aa0e4ebbe6f6f9ba/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:01.958728 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1298/2585] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:01.960415 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d288f-86d0-4ad9-9c82-e0eed09e2b12 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:01.980499 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d288f-86d0-4ad9-9c82-e0eed09e2b12 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:01.982008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8219cef1-481c-4f3c-9a49-412276149fcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=069304da63a0488880fb09c6c3af2ce6, user_id=89877030552f4954aa0e4ebbe6f6f9ba, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:01.985503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8219cef1-481c-4f3c-9a49-412276149fcf admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:01.986726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef00b8-3c1a-4f65-98c0-9f0d90385e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:01.987534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef00b8-3c1a-4f65-98c0-9f0d90385e3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:01.988087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef00b8-3c1a-4f65-98c0-9f0d90385e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:01.988255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef00b8-3c1a-4f65-98c0-9f0d90385e3c admin admin] PATH_INFO: `/v3/projects/03c28ab924ef4dc6a673a1e4f7f1a567/users/1f0d74f7dd9145aba7ac9ccf5d053905/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:02.014477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef00b8-3c1a-4f65-98c0-9f0d90385e3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=03c28ab924ef4dc6a673a1e4f7f1a567, user_id=1f0d74f7dd9145aba7ac9ccf5d053905, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:02.014684 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1288/2586] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:01 2022] PUT /identity/v3/projects/069304da63a0488880fb09c6c3af2ce6/users/89877030552f4954aa0e4ebbe6f6f9ba/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:02.025575 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34fc10-a0ad-4630-93c6-b4850a9d1ceb None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:02.025891 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34fc10-a0ad-4630-93c6-b4850a9d1ceb None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:02.026574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae34fc10-a0ad-4630-93c6-b4850a9d1ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:02.027169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef00b8-3c1a-4f65-98c0-9f0d90385e3c admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:02.062323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1299/2587] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:01 2022] PUT /identity/v3/projects/03c28ab924ef4dc6a673a1e4f7f1a567/users/1f0d74f7dd9145aba7ac9ccf5d053905/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:02.064835 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524aff72-4aec-409a-a8e2-00d23d67943b None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:02.066378 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524aff72-4aec-409a-a8e2-00d23d67943b None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:02.066819 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524aff72-4aec-409a-a8e2-00d23d67943b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:02.089204 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-ae34fc10-a0ad-4630-93c6-b4850a9d1ceb None None] MFA Rules not processed for user `89877030552f4954aa0e4ebbe6f6f9ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:02.094405 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ae34fc10-a0ad-4630-93c6-b4850a9d1ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:02.119922 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-524aff72-4aec-409a-a8e2-00d23d67943b None None] MFA Rules not processed for user `1f0d74f7dd9145aba7ac9ccf5d053905`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:02.121953 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-524aff72-4aec-409a-a8e2-00d23d67943b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:02.267907 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1289/2588] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:02.273007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1300/2589] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:02.276034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34fc10-a0ad-4630-93c6-b4850a9d1ceb None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:02.281604 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524aff72-4aec-409a-a8e2-00d23d67943b None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:02.283722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-524aff72-4aec-409a-a8e2-00d23d67943b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:02.316198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:02.387903 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae34fc10-a0ad-4630-93c6-b4850a9d1ceb None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:02.401620 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce1587-ade3-4bb1-9f7c-bbda9d4e7da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:02.402297 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce1587-ade3-4bb1-9f7c-bbda9d4e7da8 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:02.402583 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce1587-ade3-4bb1-9f7c-bbda9d4e7da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:02.402860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce1587-ade3-4bb1-9f7c-bbda9d4e7da8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:02.408795 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce1587-ade3-4bb1-9f7c-bbda9d4e7da8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:02.419124 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce1587-ade3-4bb1-9f7c-bbda9d4e7da8 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:02.459790 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524aff72-4aec-409a-a8e2-00d23d67943b None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:02.489489 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80378f0-065a-440a-b719-0bdf5725ba5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:02.490116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80378f0-065a-440a-b719-0bdf5725ba5d service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:02.490400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80378f0-065a-440a-b719-0bdf5725ba5d service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:02.490719 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80378f0-065a-440a-b719-0bdf5725ba5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:02.506747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1290/2590] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:24:02 2022] POST /identity/v3/projects => generated 387 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:24:02.508217 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce1587-ade3-4bb1-9f7c-bbda9d4e7da8 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:02.521025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80378f0-065a-440a-b719-0bdf5725ba5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:02.527057 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80378f0-065a-440a-b719-0bdf5725ba5d service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:02.550624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1301/2591] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:24:02.679706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce1587-ade3-4bb1-9f7c-bbda9d4e7da8 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:02.686818 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83309a-88de-4c99-a5c8-e9a646259efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:02.688342 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83309a-88de-4c99-a5c8-e9a646259efc admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:02.688773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83309a-88de-4c99-a5c8-e9a646259efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:02.689464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83309a-88de-4c99-a5c8-e9a646259efc admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:02.690711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83309a-88de-4c99-a5c8-e9a646259efc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:02.697316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83309a-88de-4c99-a5c8-e9a646259efc admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:02.738319 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1291/2592] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:24:02 2022] POST /identity/v3/users => generated 300 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:02.742399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80378f0-065a-440a-b719-0bdf5725ba5d service nova] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:02.760512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80378f0-065a-440a-b719-0bdf5725ba5d service nova] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:02.774156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b388e-017b-4c07-bf84-85b7deb6a261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:02.774824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b388e-017b-4c07-bf84-85b7deb6a261 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:02.775149 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b388e-017b-4c07-bf84-85b7deb6a261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:02.775436 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b388e-017b-4c07-bf84-85b7deb6a261 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:02.775999 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b388e-017b-4c07-bf84-85b7deb6a261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:02.783044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b388e-017b-4c07-bf84-85b7deb6a261 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:02.798114 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1302/2593] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:24:02.809694 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83309a-88de-4c99-a5c8-e9a646259efc admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:02.826547 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83309a-88de-4c99-a5c8-e9a646259efc admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:02.838110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebad280-4190-4ac0-ab70-1d74d1ad6fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:02.848601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebad280-4190-4ac0-ab70-1d74d1ad6fab admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:02.848601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebad280-4190-4ac0-ab70-1d74d1ad6fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:02.849621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebad280-4190-4ac0-ab70-1d74d1ad6fab admin admin] PATH_INFO: `/v3/projects/334cda1da2744fc697cd2466cc666042/users/a20bd6d4d549467594ab077acdd3a4f0/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:02.895553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebad280-4190-4ac0-ab70-1d74d1ad6fab admin admin] RBAC: Authorizing `identity:create_grant(project_id=334cda1da2744fc697cd2466cc666042, user_id=a20bd6d4d549467594ab077acdd3a4f0, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:02.898469 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebad280-4190-4ac0-ab70-1d74d1ad6fab admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:02.919159 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1292/2594] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:02 2022] PUT /identity/v3/projects/334cda1da2744fc697cd2466cc666042/users/a20bd6d4d549467594ab077acdd3a4f0/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:02.923385 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b388e-017b-4c07-bf84-85b7deb6a261 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:02.930454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:02.938201 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b388e-017b-4c07-bf84-85b7deb6a261 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:03.028710 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ae55e-3140-4a0b-9da5-dd6841d5511b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:03.029447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ae55e-3140-4a0b-9da5-dd6841d5511b admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:03.030564 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ae55e-3140-4a0b-9da5-dd6841d5511b admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:03.030564 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ae55e-3140-4a0b-9da5-dd6841d5511b admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:03.031687 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ae55e-3140-4a0b-9da5-dd6841d5511b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:03.033418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ae55e-3140-4a0b-9da5-dd6841d5511b admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:03.050349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1303/2595] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:03.051178 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebad280-4190-4ac0-ab70-1d74d1ad6fab admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:03.075001 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebad280-4190-4ac0-ab70-1d74d1ad6fab admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:03.088047 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461ea7d-623d-4e53-a180-54426aeef633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:03.088922 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461ea7d-623d-4e53-a180-54426aeef633 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:03.089590 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461ea7d-623d-4e53-a180-54426aeef633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:03.089877 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5461ea7d-623d-4e53-a180-54426aeef633 admin admin] PATH_INFO: `/v3/projects/334cda1da2744fc697cd2466cc666042/users/a20bd6d4d549467594ab077acdd3a4f0/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:03.095425 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461ea7d-623d-4e53-a180-54426aeef633 admin admin] RBAC: Authorizing `identity:create_grant(project_id=334cda1da2744fc697cd2466cc666042, user_id=a20bd6d4d549467594ab077acdd3a4f0, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:03.103241 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5461ea7d-623d-4e53-a180-54426aeef633 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:03.128722 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1293/2596] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:03 2022] PUT /identity/v3/projects/334cda1da2744fc697cd2466cc666042/users/a20bd6d4d549467594ab077acdd3a4f0/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:03.140995 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e48d2-df90-4350-9750-e229da9eb2f5 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:03.141251 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e48d2-df90-4350-9750-e229da9eb2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:03.141528 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e48d2-df90-4350-9750-e229da9eb2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:03.203856 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-181e48d2-df90-4350-9750-e229da9eb2f5 None None] MFA Rules not processed for user `a20bd6d4d549467594ab077acdd3a4f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:03.205109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-181e48d2-df90-4350-9750-e229da9eb2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:03.366821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1304/2597] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:03 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:03.378644 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461ea7d-623d-4e53-a180-54426aeef633 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:03.380474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-5461ea7d-623d-4e53-a180-54426aeef633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:03.474243 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5461ea7d-623d-4e53-a180-54426aeef633 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:03.478261 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec95c2-4479-44e2-8117-897a6793d776 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:03.479360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec95c2-4479-44e2-8117-897a6793d776 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:03.479360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec95c2-4479-44e2-8117-897a6793d776 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:03.479651 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec95c2-4479-44e2-8117-897a6793d776 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:03.491204 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec95c2-4479-44e2-8117-897a6793d776 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:03.492665 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec95c2-4479-44e2-8117-897a6793d776 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:03.498807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:03.505299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1294/2598] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:03.788302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e48d2-df90-4350-9750-e229da9eb2f5 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:03.916468 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e48d2-df90-4350-9750-e229da9eb2f5 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:03.922535 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e4246-03a9-4d84-b9f3-c0242b94343a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:03.923400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e4246-03a9-4d84-b9f3-c0242b94343a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:03.923598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e4246-03a9-4d84-b9f3-c0242b94343a admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:03.924035 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e4246-03a9-4d84-b9f3-c0242b94343a admin admin] PATH_INFO: `/v3/users/1f0d74f7dd9145aba7ac9ccf5d053905` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:03.925550 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e4246-03a9-4d84-b9f3-c0242b94343a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f0d74f7dd9145aba7ac9ccf5d053905)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:03.927478 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e4246-03a9-4d84-b9f3-c0242b94343a admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:03.965499 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:04.017227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d90e4246-03a9-4d84-b9f3-c0242b94343a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f0d74f7dd9145aba7ac9ccf5d053905'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:04.026453 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d90e4246-03a9-4d84-b9f3-c0242b94343a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f0d74f7dd9145aba7ac9ccf5d053905'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:04.027837 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d90e4246-03a9-4d84-b9f3-c0242b94343a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f0d74f7dd9145aba7ac9ccf5d053905'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:04.042138 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d90e4246-03a9-4d84-b9f3-c0242b94343a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f0d74f7dd9145aba7ac9ccf5d053905'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:04.043018 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-d90e4246-03a9-4d84-b9f3-c0242b94343a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f0d74f7dd9145aba7ac9ccf5d053905'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:04.058068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1305/2599] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:24:03 2022] DELETE /identity/v3/users/1f0d74f7dd9145aba7ac9ccf5d053905 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:04.063345 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82987a4f-1fda-4834-957f-daf138edd8c7 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:04.063835 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82987a4f-1fda-4834-957f-daf138edd8c7 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:04.064291 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82987a4f-1fda-4834-957f-daf138edd8c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:04.085743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-82987a4f-1fda-4834-957f-daf138edd8c7 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:04.087457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-82987a4f-1fda-4834-957f-daf138edd8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:04.169279 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1295/2600] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:24:04.186788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e4246-03a9-4d84-b9f3-c0242b94343a admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:04.191201 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-d90e4246-03a9-4d84-b9f3-c0242b94343a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:04.284230 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e4246-03a9-4d84-b9f3-c0242b94343a admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:04.288365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427b967-613a-4b83-8059-9bc4b96e7068 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:04.289125 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427b967-613a-4b83-8059-9bc4b96e7068 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:04.289479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427b967-613a-4b83-8059-9bc4b96e7068 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:04.289968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427b967-613a-4b83-8059-9bc4b96e7068 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:04.301269 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427b967-613a-4b83-8059-9bc4b96e7068 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:04.303894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f427b967-613a-4b83-8059-9bc4b96e7068 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:04.315850 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1306/2601] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:05.136246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82987a4f-1fda-4834-957f-daf138edd8c7 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:05.138900 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-82987a4f-1fda-4834-957f-daf138edd8c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:05.152600 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82987a4f-1fda-4834-957f-daf138edd8c7 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:05.167594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d72b5f-282d-4b31-b66b-e10492c9a14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:05.168332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d72b5f-282d-4b31-b66b-e10492c9a14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:05.168860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d72b5f-282d-4b31-b66b-e10492c9a14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:05.169447 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d72b5f-282d-4b31-b66b-e10492c9a14d admin admin] PATH_INFO: `/v3/projects/03c28ab924ef4dc6a673a1e4f7f1a567` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:05.170839 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d72b5f-282d-4b31-b66b-e10492c9a14d admin admin] RBAC: Authorizing `identity:delete_project(project_id=03c28ab924ef4dc6a673a1e4f7f1a567)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:05.173746 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d72b5f-282d-4b31-b66b-e10492c9a14d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:05.272509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-16d72b5f-282d-4b31-b66b-e10492c9a14d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03c28ab924ef4dc6a673a1e4f7f1a567', 'request_id': 'req-16d72b5f-282d-4b31-b66b-e10492c9a14d'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:05.293488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-16d72b5f-282d-4b31-b66b-e10492c9a14d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03c28ab924ef4dc6a673a1e4f7f1a567', 'request_id': 'req-16d72b5f-282d-4b31-b66b-e10492c9a14d'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:05.301873 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-16d72b5f-282d-4b31-b66b-e10492c9a14d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03c28ab924ef4dc6a673a1e4f7f1a567', 'request_id': 'req-16d72b5f-282d-4b31-b66b-e10492c9a14d'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:05.318438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-16d72b5f-282d-4b31-b66b-e10492c9a14d admin admin] The token cache is being invalidate because project 03c28ab924ef4dc6a673a1e4f7f1a567 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:24:05.318741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-16d72b5f-282d-4b31-b66b-e10492c9a14d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:05.320078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-16d72b5f-282d-4b31-b66b-e10492c9a14d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:05.323022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1296/2602] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:24:05 2022] DELETE /identity/v3/projects/03c28ab924ef4dc6a673a1e4f7f1a567 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:05.332673 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427b967-613a-4b83-8059-9bc4b96e7068 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:05.347501 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427b967-613a-4b83-8059-9bc4b96e7068 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:05.483093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8777f8-60e3-4dda-8c75-4f46660867f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:05.484045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8777f8-60e3-4dda-8c75-4f46660867f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:05.484552 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8777f8-60e3-4dda-8c75-4f46660867f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:05.485007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8777f8-60e3-4dda-8c75-4f46660867f6 admin admin] PATH_INFO: `/v3/users/a20bd6d4d549467594ab077acdd3a4f0` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:05.487134 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8777f8-60e3-4dda-8c75-4f46660867f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a20bd6d4d549467594ab077acdd3a4f0)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:05.490303 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8777f8-60e3-4dda-8c75-4f46660867f6 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:05.559663 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-ae8777f8-60e3-4dda-8c75-4f46660867f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a20bd6d4d549467594ab077acdd3a4f0'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:05.560906 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:05.571077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-ae8777f8-60e3-4dda-8c75-4f46660867f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a20bd6d4d549467594ab077acdd3a4f0'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:05.575973 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-ae8777f8-60e3-4dda-8c75-4f46660867f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a20bd6d4d549467594ab077acdd3a4f0'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:05.600697 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-ae8777f8-60e3-4dda-8c75-4f46660867f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a20bd6d4d549467594ab077acdd3a4f0'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:05.601679 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-ae8777f8-60e3-4dda-8c75-4f46660867f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a20bd6d4d549467594ab077acdd3a4f0'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:05.634416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1307/2603] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:24:05 2022] DELETE /identity/v3/users/a20bd6d4d549467594ab077acdd3a4f0 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:05.842064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d72b5f-282d-4b31-b66b-e10492c9a14d admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:05.843487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-16d72b5f-282d-4b31-b66b-e10492c9a14d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:05.881067 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d72b5f-282d-4b31-b66b-e10492c9a14d admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:06.053208 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bb33b-8033-428f-9059-1e2fc9191de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:06.055911 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bb33b-8033-428f-9059-1e2fc9191de0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:06.056522 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bb33b-8033-428f-9059-1e2fc9191de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:06.057526 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bb33b-8033-428f-9059-1e2fc9191de0 admin admin] PATH_INFO: `/v3/users/89877030552f4954aa0e4ebbe6f6f9ba` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:06.060491 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bb33b-8033-428f-9059-1e2fc9191de0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89877030552f4954aa0e4ebbe6f6f9ba)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:06.078880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bb33b-8033-428f-9059-1e2fc9191de0 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:06.132828 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:06.167765 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-476bb33b-8033-428f-9059-1e2fc9191de0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89877030552f4954aa0e4ebbe6f6f9ba'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:06.168488 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-476bb33b-8033-428f-9059-1e2fc9191de0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89877030552f4954aa0e4ebbe6f6f9ba'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:06.169294 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-476bb33b-8033-428f-9059-1e2fc9191de0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89877030552f4954aa0e4ebbe6f6f9ba'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:06.188601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-476bb33b-8033-428f-9059-1e2fc9191de0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89877030552f4954aa0e4ebbe6f6f9ba'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:06.197433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-476bb33b-8033-428f-9059-1e2fc9191de0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89877030552f4954aa0e4ebbe6f6f9ba'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:06.220668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1297/2604] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:24:05 2022] DELETE /identity/v3/users/89877030552f4954aa0e4ebbe6f6f9ba => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:06.225031 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11029b98-45bb-47d2-8569-d789f3a1f440 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:06.225683 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11029b98-45bb-47d2-8569-d789f3a1f440 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:06.226141 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11029b98-45bb-47d2-8569-d789f3a1f440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:06.250387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-11029b98-45bb-47d2-8569-d789f3a1f440 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:06.251915 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-11029b98-45bb-47d2-8569-d789f3a1f440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:06.338247 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1308/2605] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:06.350382 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bb33b-8033-428f-9059-1e2fc9191de0 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:06.351963 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-476bb33b-8033-428f-9059-1e2fc9191de0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:06.492747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bb33b-8033-428f-9059-1e2fc9191de0 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:06.499091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a948ac-7be7-41c1-816e-9342254e78c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:06.500117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a948ac-7be7-41c1-816e-9342254e78c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:06.500452 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a948ac-7be7-41c1-816e-9342254e78c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:06.500819 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a948ac-7be7-41c1-816e-9342254e78c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:06.539986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a948ac-7be7-41c1-816e-9342254e78c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:06.544348 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a948ac-7be7-41c1-816e-9342254e78c5 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:06.580722 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1298/2606] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:06.726652 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11029b98-45bb-47d2-8569-d789f3a1f440 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:06.731400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-11029b98-45bb-47d2-8569-d789f3a1f440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:06.749086 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11029b98-45bb-47d2-8569-d789f3a1f440 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:06.759268 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbeddf0-d6f1-4a6a-9705-a46f2f5528df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:06.760232 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeddf0-d6f1-4a6a-9705-a46f2f5528df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:06.760716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeddf0-d6f1-4a6a-9705-a46f2f5528df admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:06.761551 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbeddf0-d6f1-4a6a-9705-a46f2f5528df admin admin] PATH_INFO: `/v3/projects/334cda1da2744fc697cd2466cc666042` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:06.763707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbeddf0-d6f1-4a6a-9705-a46f2f5528df admin admin] RBAC: Authorizing `identity:delete_project(project_id=334cda1da2744fc697cd2466cc666042)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:06.766926 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbeddf0-d6f1-4a6a-9705-a46f2f5528df admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:06.909272 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6dbeddf0-d6f1-4a6a-9705-a46f2f5528df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '334cda1da2744fc697cd2466cc666042', 'request_id': 'req-6dbeddf0-d6f1-4a6a-9705-a46f2f5528df'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:06.939676 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6dbeddf0-d6f1-4a6a-9705-a46f2f5528df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '334cda1da2744fc697cd2466cc666042', 'request_id': 'req-6dbeddf0-d6f1-4a6a-9705-a46f2f5528df'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:06.948039 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6dbeddf0-d6f1-4a6a-9705-a46f2f5528df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '334cda1da2744fc697cd2466cc666042', 'request_id': 'req-6dbeddf0-d6f1-4a6a-9705-a46f2f5528df'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:06.979488 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6dbeddf0-d6f1-4a6a-9705-a46f2f5528df admin admin] The token cache is being invalidate because project 334cda1da2744fc697cd2466cc666042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:24:06.980585 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6dbeddf0-d6f1-4a6a-9705-a46f2f5528df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:06.981421 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6dbeddf0-d6f1-4a6a-9705-a46f2f5528df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:06.988592 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d566e-e779-4ba8-bd1e-c1767ec262fa None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:06.989149 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d566e-e779-4ba8-bd1e-c1767ec262fa None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:06.989527 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565d566e-e779-4ba8-bd1e-c1767ec262fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:06.995541 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1309/2607] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:24:06 2022] DELETE /identity/v3/projects/334cda1da2744fc697cd2466cc666042 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:07.016818 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-565d566e-e779-4ba8-bd1e-c1767ec262fa None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:07.021714 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-565d566e-e779-4ba8-bd1e-c1767ec262fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:07.246578 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1299/2608] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:07.255228 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97dc5a-faca-403e-900a-01db954ecb0e None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:07.255755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97dc5a-faca-403e-900a-01db954ecb0e None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:07.256184 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97dc5a-faca-403e-900a-01db954ecb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:07.310469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-5f97dc5a-faca-403e-900a-01db954ecb0e None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:07.318505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-5f97dc5a-faca-403e-900a-01db954ecb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:07.332810 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1310/2609] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:07.336968 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d566e-e779-4ba8-bd1e-c1767ec262fa None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:07.356277 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565d566e-e779-4ba8-bd1e-c1767ec262fa None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:07.362542 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575d16a-c724-4fba-8ee7-298bc2b99fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:07.363672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575d16a-c724-4fba-8ee7-298bc2b99fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:07.364182 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575d16a-c724-4fba-8ee7-298bc2b99fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:07.364838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575d16a-c724-4fba-8ee7-298bc2b99fd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:07.372916 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575d16a-c724-4fba-8ee7-298bc2b99fd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:24:07.374160 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575d16a-c724-4fba-8ee7-298bc2b99fd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:07.376503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575d16a-c724-4fba-8ee7-298bc2b99fd4 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:07.392145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1300/2610] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:24:07.397398 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97dc5a-faca-403e-900a-01db954ecb0e None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:07.408165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97dc5a-faca-403e-900a-01db954ecb0e None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:07.426753 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877359ed-c5a1-4493-8094-b361e8f12c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:07.427800 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877359ed-c5a1-4493-8094-b361e8f12c5b admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:07.428320 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877359ed-c5a1-4493-8094-b361e8f12c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:07.429082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877359ed-c5a1-4493-8094-b361e8f12c5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:07.432512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877359ed-c5a1-4493-8094-b361e8f12c5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:07.442078 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877359ed-c5a1-4493-8094-b361e8f12c5b admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:07.492137 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575d16a-c724-4fba-8ee7-298bc2b99fd4 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:07.494582 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-6575d16a-c724-4fba-8ee7-298bc2b99fd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:07.500537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1311/2611] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:24:07 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:07.503717 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877359ed-c5a1-4493-8094-b361e8f12c5b admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:07.550880 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:07.595606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575d16a-c724-4fba-8ee7-298bc2b99fd4 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:07.677260 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877359ed-c5a1-4493-8094-b361e8f12c5b admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:07.690944 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:07.692019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63890394-ba10-48a1-bb81-6c5f498f1d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:07.692887 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63890394-ba10-48a1-bb81-6c5f498f1d93 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:07.693197 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63890394-ba10-48a1-bb81-6c5f498f1d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:07.693624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63890394-ba10-48a1-bb81-6c5f498f1d93 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:07.694306 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63890394-ba10-48a1-bb81-6c5f498f1d93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:07.696703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63890394-ba10-48a1-bb81-6c5f498f1d93 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:07.738424 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1312/2612] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:24:07 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:07.743705 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63890394-ba10-48a1-bb81-6c5f498f1d93 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:07.765803 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63890394-ba10-48a1-bb81-6c5f498f1d93 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:07.766249 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a5ca8-1b5c-46b3-900a-777c094c77d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:07.766918 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a5ca8-1b5c-46b3-900a-777c094c77d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:07.767199 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a5ca8-1b5c-46b3-900a-777c094c77d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:07.767472 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a5ca8-1b5c-46b3-900a-777c094c77d0 admin admin] PATH_INFO: `/v3/projects/069304da63a0488880fb09c6c3af2ce6` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:07.768734 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a5ca8-1b5c-46b3-900a-777c094c77d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=069304da63a0488880fb09c6c3af2ce6)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:07.772744 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a5ca8-1b5c-46b3-900a-777c094c77d0 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:07.778951 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9748c47-1a77-4206-889f-748f5c194b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:07.779724 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9748c47-1a77-4206-889f-748f5c194b5c admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:07.780027 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9748c47-1a77-4206-889f-748f5c194b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:07.782154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9748c47-1a77-4206-889f-748f5c194b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:07.785081 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9748c47-1a77-4206-889f-748f5c194b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:07.786817 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9748c47-1a77-4206-889f-748f5c194b5c admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:07.816897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1313/2613] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:07.820885 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9748c47-1a77-4206-889f-748f5c194b5c admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:07.844435 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9748c47-1a77-4206-889f-748f5c194b5c admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:07.876279 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-be8a5ca8-1b5c-46b3-900a-777c094c77d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '069304da63a0488880fb09c6c3af2ce6', 'request_id': 'req-be8a5ca8-1b5c-46b3-900a-777c094c77d0'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:07.895949 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-be8a5ca8-1b5c-46b3-900a-777c094c77d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '069304da63a0488880fb09c6c3af2ce6', 'request_id': 'req-be8a5ca8-1b5c-46b3-900a-777c094c77d0'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:07.910031 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-be8a5ca8-1b5c-46b3-900a-777c094c77d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '069304da63a0488880fb09c6c3af2ce6', 'request_id': 'req-be8a5ca8-1b5c-46b3-900a-777c094c77d0'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:07.939852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-be8a5ca8-1b5c-46b3-900a-777c094c77d0 admin admin] The token cache is being invalidate because project 069304da63a0488880fb09c6c3af2ce6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:24:07.940145 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-be8a5ca8-1b5c-46b3-900a-777c094c77d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:07.944738 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-be8a5ca8-1b5c-46b3-900a-777c094c77d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:07.948724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1301/2614] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:24:07 2022] DELETE /identity/v3/projects/069304da63a0488880fb09c6c3af2ce6 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:24:07.954431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e016e2d-13bf-41ec-8c4e-4a53bca2d5b1 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:07.954717 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e016e2d-13bf-41ec-8c4e-4a53bca2d5b1 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:07.954942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e016e2d-13bf-41ec-8c4e-4a53bca2d5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:07.979717 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fa7b1-04f1-476c-ab57-cbd4318a7189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:07.980577 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fa7b1-04f1-476c-ab57-cbd4318a7189 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:07.981011 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fa7b1-04f1-476c-ab57-cbd4318a7189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:07.981379 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fa7b1-04f1-476c-ab57-cbd4318a7189 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:07.982157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fa7b1-04f1-476c-ab57-cbd4318a7189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:07.984190 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fa7b1-04f1-476c-ab57-cbd4318a7189 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:08.006453 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1314/2615] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:08.007618 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fa7b1-04f1-476c-ab57-cbd4318a7189 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:08.009881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-7e016e2d-13bf-41ec-8c4e-4a53bca2d5b1 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:08.010082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-893fa7b1-04f1-476c-ab57-cbd4318a7189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:08.011307 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7e016e2d-13bf-41ec-8c4e-4a53bca2d5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:08.025556 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1302/2616] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:24:08.034558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b8b62-73ab-4132-b5b9-a48c7800ad5c None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:08.035073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b8b62-73ab-4132-b5b9-a48c7800ad5c None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:08.035451 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b8b62-73ab-4132-b5b9-a48c7800ad5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:08.040362 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fa7b1-04f1-476c-ab57-cbd4318a7189 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:08.061126 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8e699-82ec-4347-8812-29b74f923561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:08.062770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8e699-82ec-4347-8812-29b74f923561 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:08.063223 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8e699-82ec-4347-8812-29b74f923561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:08.063653 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8e699-82ec-4347-8812-29b74f923561 admin admin] PATH_INFO: `/v3/projects/cc6ac4a9576b4e4abf1a23eaa5a2b735/users/6e38e40018474d7098023745a1052f74/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:08.072705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-cf4b8b62-73ab-4132-b5b9-a48c7800ad5c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:08.072705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-cf4b8b62-73ab-4132-b5b9-a48c7800ad5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:08.093433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1303/2617] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:08.104410 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b8b62-73ab-4132-b5b9-a48c7800ad5c None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:08.115368 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e8e699-82ec-4347-8812-29b74f923561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc6ac4a9576b4e4abf1a23eaa5a2b735, user_id=6e38e40018474d7098023745a1052f74, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:08.120647 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e8e699-82ec-4347-8812-29b74f923561 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:08.128572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b8b62-73ab-4132-b5b9-a48c7800ad5c None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:08.136476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00675a-4051-4ddd-8950-27ba5d665b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:08.139572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00675a-4051-4ddd-8950-27ba5d665b36 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:08.140980 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00675a-4051-4ddd-8950-27ba5d665b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:08.141452 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00675a-4051-4ddd-8950-27ba5d665b36 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:08.142278 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00675a-4051-4ddd-8950-27ba5d665b36 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:24:08.144271 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00675a-4051-4ddd-8950-27ba5d665b36 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:08.146840 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00675a-4051-4ddd-8950-27ba5d665b36 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:08.156720 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1315/2618] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:08 2022] PUT /identity/v3/projects/cc6ac4a9576b4e4abf1a23eaa5a2b735/users/6e38e40018474d7098023745a1052f74/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:08.170192 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b06cb6-21dc-4c4e-a26c-3bb281750b59 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:08.170832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b06cb6-21dc-4c4e-a26c-3bb281750b59 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:08.171991 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b06cb6-21dc-4c4e-a26c-3bb281750b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:08.194581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1304/2619] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:24:08.215131 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00675a-4051-4ddd-8950-27ba5d665b36 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:08.252647 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00675a-4051-4ddd-8950-27ba5d665b36 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:08.423416 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87346f84-d3da-4c30-b538-c4bd603dacba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:08.425338 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87346f84-d3da-4c30-b538-c4bd603dacba admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:08.425635 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87346f84-d3da-4c30-b538-c4bd603dacba admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:08.425917 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87346f84-d3da-4c30-b538-c4bd603dacba admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:08.426706 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87346f84-d3da-4c30-b538-c4bd603dacba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:08.429220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87346f84-d3da-4c30-b538-c4bd603dacba admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:08.520972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c7b06cb6-21dc-4c4e-a26c-3bb281750b59 None None] MFA Rules not processed for user `6e38e40018474d7098023745a1052f74`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:08.522962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c7b06cb6-21dc-4c4e-a26c-3bb281750b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:08.526148 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1305/2620] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:24:08 2022] POST /identity/v3/projects => generated 371 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:08.527287 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87346f84-d3da-4c30-b538-c4bd603dacba admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:08.574677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:08.684334 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87346f84-d3da-4c30-b538-c4bd603dacba admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:08.700101 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:08.700352 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a011c14-2ecb-4b13-a45c-e776a998a25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:08.701200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a011c14-2ecb-4b13-a45c-e776a998a25f admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:08.701711 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a011c14-2ecb-4b13-a45c-e776a998a25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:08.701989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a011c14-2ecb-4b13-a45c-e776a998a25f admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:08.703469 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a011c14-2ecb-4b13-a45c-e776a998a25f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:08.705435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a011c14-2ecb-4b13-a45c-e776a998a25f admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:08.714512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1316/2621] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 542 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:08.744311 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b06cb6-21dc-4c4e-a26c-3bb281750b59 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:08.747331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c7b06cb6-21dc-4c4e-a26c-3bb281750b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:08.758106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1306/2622] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:24:08 2022] POST /identity/v3/users => generated 286 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:08.762742 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a011c14-2ecb-4b13-a45c-e776a998a25f admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:08.783114 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a011c14-2ecb-4b13-a45c-e776a998a25f admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:08.799325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae677c0e-2914-4e62-a18b-8d2880ee8227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:08.800028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae677c0e-2914-4e62-a18b-8d2880ee8227 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:08.800388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae677c0e-2914-4e62-a18b-8d2880ee8227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:08.800824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae677c0e-2914-4e62-a18b-8d2880ee8227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:08.801553 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae677c0e-2914-4e62-a18b-8d2880ee8227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:08.803229 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae677c0e-2914-4e62-a18b-8d2880ee8227 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:08.843317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1307/2623] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:24:08.848897 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae677c0e-2914-4e62-a18b-8d2880ee8227 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:08.868305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b06cb6-21dc-4c4e-a26c-3bb281750b59 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:08.874086 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae677c0e-2914-4e62-a18b-8d2880ee8227 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:08.878439 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2215f-0357-4346-9da5-010363c39744 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:08.879503 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2215f-0357-4346-9da5-010363c39744 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:08.887565 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2215f-0357-4346-9da5-010363c39744 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:08.888094 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2215f-0357-4346-9da5-010363c39744 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:08.890786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5317a0-3c06-4c4d-a20a-f64e51e17468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:08.891415 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5317a0-3c06-4c4d-a20a-f64e51e17468 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:08.891642 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5317a0-3c06-4c4d-a20a-f64e51e17468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:08.891928 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5317a0-3c06-4c4d-a20a-f64e51e17468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:08.892803 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5317a0-3c06-4c4d-a20a-f64e51e17468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:08.894425 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5317a0-3c06-4c4d-a20a-f64e51e17468 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:08.901678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2215f-0357-4346-9da5-010363c39744 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:08.911331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2215f-0357-4346-9da5-010363c39744 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:08.919487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1308/2624] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:08.924558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5317a0-3c06-4c4d-a20a-f64e51e17468 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:08.943324 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1317/2625] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:08.954507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5317a0-3c06-4c4d-a20a-f64e51e17468 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:08.963601 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e984d-2580-48a7-b35d-2f2f9f808b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:08.964286 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e984d-2580-48a7-b35d-2f2f9f808b54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:08.965836 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e984d-2580-48a7-b35d-2f2f9f808b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:08.966125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e984d-2580-48a7-b35d-2f2f9f808b54 admin admin] PATH_INFO: `/v3/projects/a8d813d819034349b6fac43872a5d91e/users/8206a2d308ab42f3ba5908bd635a38e3/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:08.987403 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e984d-2580-48a7-b35d-2f2f9f808b54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d813d819034349b6fac43872a5d91e, user_id=8206a2d308ab42f3ba5908bd635a38e3, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:08.990275 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e984d-2580-48a7-b35d-2f2f9f808b54 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:09.016436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1309/2626] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:08 2022] PUT /identity/v3/projects/a8d813d819034349b6fac43872a5d91e/users/8206a2d308ab42f3ba5908bd635a38e3/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:09.028232 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c4e15-ba0b-4be2-81fd-6cf77ae5eb78 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:09.028519 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c4e15-ba0b-4be2-81fd-6cf77ae5eb78 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:09.028878 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c4e15-ba0b-4be2-81fd-6cf77ae5eb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:09.064286 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582cacc-7af4-422d-a7db-7afb48f0de67 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:09.064590 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582cacc-7af4-422d-a7db-7afb48f0de67 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:09.064888 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582cacc-7af4-422d-a7db-7afb48f0de67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:09.080652 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c21c4e15-ba0b-4be2-81fd-6cf77ae5eb78 None None] MFA Rules not processed for user `8206a2d308ab42f3ba5908bd635a38e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:09.087491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c21c4e15-ba0b-4be2-81fd-6cf77ae5eb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:09.135431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-d582cacc-7af4-422d-a7db-7afb48f0de67 None None] MFA Rules not processed for user `fb3a977fea2c499eb915bc059ab01a79`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:09.141090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d582cacc-7af4-422d-a7db-7afb48f0de67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:09.220557 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1318/2627] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:09.226174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c4e15-ba0b-4be2-81fd-6cf77ae5eb78 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:09.340824 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1310/2628] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:24:09.342768 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582cacc-7af4-422d-a7db-7afb48f0de67 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:09.345535 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d582cacc-7af4-422d-a7db-7afb48f0de67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:09.384514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c4e15-ba0b-4be2-81fd-6cf77ae5eb78 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:09.389916 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fdf81d-15e4-43ad-be45-26db1c26d062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:09.390731 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fdf81d-15e4-43ad-be45-26db1c26d062 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:09.391017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fdf81d-15e4-43ad-be45-26db1c26d062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:09.391476 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fdf81d-15e4-43ad-be45-26db1c26d062 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:09.392168 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fdf81d-15e4-43ad-be45-26db1c26d062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:09.394633 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fdf81d-15e4-43ad-be45-26db1c26d062 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:09.459154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1319/2629] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:24:09 2022] POST /identity/v3/projects => generated 373 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:24:09.466763 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fdf81d-15e4-43ad-be45-26db1c26d062 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:09.512937 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:09.538758 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:09.565180 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582cacc-7af4-422d-a7db-7afb48f0de67 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:09.604722 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fdf81d-15e4-43ad-be45-26db1c26d062 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:09.610637 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c42893-8920-41e1-885b-6009e124a597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:09.611282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c42893-8920-41e1-885b-6009e124a597 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:09.611560 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c42893-8920-41e1-885b-6009e124a597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:09.612608 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c42893-8920-41e1-885b-6009e124a597 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:09.612608 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c42893-8920-41e1-885b-6009e124a597 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:09.614398 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c42893-8920-41e1-885b-6009e124a597 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:09.646170 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1320/2630] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:24:09 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:09.646376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221317c9-a44f-4970-a11a-86549ef4dd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:09.646376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221317c9-a44f-4970-a11a-86549ef4dd6e admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:09.646376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221317c9-a44f-4970-a11a-86549ef4dd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:09.646376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221317c9-a44f-4970-a11a-86549ef4dd6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:09.646736 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221317c9-a44f-4970-a11a-86549ef4dd6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:09.647993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221317c9-a44f-4970-a11a-86549ef4dd6e admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:09.650019 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c42893-8920-41e1-885b-6009e124a597 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:09.666250 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c42893-8920-41e1-885b-6009e124a597 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:09.674430 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2acf334-6e1f-4fc4-88e5-88fa975abb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:09.675074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acf334-6e1f-4fc4-88e5-88fa975abb9e admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:09.675359 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acf334-6e1f-4fc4-88e5-88fa975abb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:09.675635 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2acf334-6e1f-4fc4-88e5-88fa975abb9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:09.676346 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2acf334-6e1f-4fc4-88e5-88fa975abb9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:09.678017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2acf334-6e1f-4fc4-88e5-88fa975abb9e admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:09.692293 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1311/2631] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:24:09 2022] POST /identity/v3/projects => generated 383 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:09.692907 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1321/2632] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:09.693804 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2acf334-6e1f-4fc4-88e5-88fa975abb9e admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:09.693989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221317c9-a44f-4970-a11a-86549ef4dd6e admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:09.709469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2acf334-6e1f-4fc4-88e5-88fa975abb9e admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:09.806472 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b129d0a-cb2d-4f5d-8cb8-f914b19ee243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:09.807832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b129d0a-cb2d-4f5d-8cb8-f914b19ee243 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:09.807832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b129d0a-cb2d-4f5d-8cb8-f914b19ee243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:09.807832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b129d0a-cb2d-4f5d-8cb8-f914b19ee243 admin admin] PATH_INFO: `/v3/projects/c21f8b982e9d466ea00e47b8f3008aa7/users/6b6450f1dab24181aefccf2787dbb293/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:09.814800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221317c9-a44f-4970-a11a-86549ef4dd6e admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:09.830515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379a22a-871c-40ec-b7c6-fffccd63760a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:09.831944 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379a22a-871c-40ec-b7c6-fffccd63760a admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:09.832543 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379a22a-871c-40ec-b7c6-fffccd63760a admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:09.833289 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379a22a-871c-40ec-b7c6-fffccd63760a admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:09.833824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b129d0a-cb2d-4f5d-8cb8-f914b19ee243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c21f8b982e9d466ea00e47b8f3008aa7, user_id=6b6450f1dab24181aefccf2787dbb293, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:09.834915 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379a22a-871c-40ec-b7c6-fffccd63760a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:09.836626 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b129d0a-cb2d-4f5d-8cb8-f914b19ee243 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:09.837825 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379a22a-871c-40ec-b7c6-fffccd63760a admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:09.853985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1322/2633] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:09 2022] PUT /identity/v3/projects/c21f8b982e9d466ea00e47b8f3008aa7/users/6b6450f1dab24181aefccf2787dbb293/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:09.858580 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b129d0a-cb2d-4f5d-8cb8-f914b19ee243 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:09.870325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1312/2634] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:24:09 2022] POST /identity/v3/users => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:09.871589 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379a22a-871c-40ec-b7c6-fffccd63760a admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:09.882843 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379a22a-871c-40ec-b7c6-fffccd63760a admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:09.936405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b129d0a-cb2d-4f5d-8cb8-f914b19ee243 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:09.942337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c2b55-1e68-4d5e-83f9-89c40ab852ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:09.942969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c2b55-1e68-4d5e-83f9-89c40ab852ac admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:09.945211 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c2b55-1e68-4d5e-83f9-89c40ab852ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:09.945499 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c2b55-1e68-4d5e-83f9-89c40ab852ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:09.946554 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c2b55-1e68-4d5e-83f9-89c40ab852ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:09.948457 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39c2b55-1e68-4d5e-83f9-89c40ab852ac admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:09.966866 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1323/2635] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:24:09.967866 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c2b55-1e68-4d5e-83f9-89c40ab852ac admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:09.970823 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828fbe0-dbeb-46ba-949d-b8ec7dec2901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:09.971603 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828fbe0-dbeb-46ba-949d-b8ec7dec2901 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:09.971936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828fbe0-dbeb-46ba-949d-b8ec7dec2901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:09.972272 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b828fbe0-dbeb-46ba-949d-b8ec7dec2901 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:09.973215 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828fbe0-dbeb-46ba-949d-b8ec7dec2901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:09.975731 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b828fbe0-dbeb-46ba-949d-b8ec7dec2901 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:09.984228 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c2b55-1e68-4d5e-83f9-89c40ab852ac admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:09.988580 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1313/2636] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:09.990224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d882360-7213-45cb-891a-614479a9c630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:09.991018 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d882360-7213-45cb-891a-614479a9c630 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:09.991456 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d882360-7213-45cb-891a-614479a9c630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:09.991874 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d882360-7213-45cb-891a-614479a9c630 admin admin] PATH_INFO: `/v3/projects/c21f8b982e9d466ea00e47b8f3008aa7/users/6b6450f1dab24181aefccf2787dbb293/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:09.993483 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828fbe0-dbeb-46ba-949d-b8ec7dec2901 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:09.996918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d882360-7213-45cb-891a-614479a9c630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c21f8b982e9d466ea00e47b8f3008aa7, user_id=6b6450f1dab24181aefccf2787dbb293, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:10.000667 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d882360-7213-45cb-891a-614479a9c630 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:10.004940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b828fbe0-dbeb-46ba-949d-b8ec7dec2901 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:10.010490 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad0426-f3f7-4510-9d98-0187c7dd7765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:10.011176 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad0426-f3f7-4510-9d98-0187c7dd7765 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:10.011457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad0426-f3f7-4510-9d98-0187c7dd7765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:10.011732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad0426-f3f7-4510-9d98-0187c7dd7765 admin admin] PATH_INFO: `/v3/projects/72c769241feb4f20a536bf1cb3b8a13b/users/46053624901c4b55acb1089e9dbbb5ff/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:10.013805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1324/2637] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:09 2022] PUT /identity/v3/projects/c21f8b982e9d466ea00e47b8f3008aa7/users/6b6450f1dab24181aefccf2787dbb293/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:10.014716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d882360-7213-45cb-891a-614479a9c630 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:10.016975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-0d882360-7213-45cb-891a-614479a9c630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:10.025886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:10.029712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:10.032438 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad0426-f3f7-4510-9d98-0187c7dd7765 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72c769241feb4f20a536bf1cb3b8a13b, user_id=46053624901c4b55acb1089e9dbbb5ff, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:10.036254 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad0426-f3f7-4510-9d98-0187c7dd7765 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:10.052401 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1314/2638] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:09 2022] PUT /identity/v3/projects/72c769241feb4f20a536bf1cb3b8a13b/users/46053624901c4b55acb1089e9dbbb5ff/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:10.053860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891f821-d240-44e6-8a7d-7543a32c37fa None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:10.054305 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891f821-d240-44e6-8a7d-7543a32c37fa None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:10.054618 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b891f821-d240-44e6-8a7d-7543a32c37fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:10.084687 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-b891f821-d240-44e6-8a7d-7543a32c37fa None None] MFA Rules not processed for user `6b6450f1dab24181aefccf2787dbb293`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:10.086288 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-b891f821-d240-44e6-8a7d-7543a32c37fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:10.111382 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d882360-7213-45cb-891a-614479a9c630 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:10.186593 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1315/2639] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:10.188207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891f821-d240-44e6-8a7d-7543a32c37fa None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:10.210701 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45a3e9-9d4a-4376-90f5-ea5184cb12bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:10.211348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45a3e9-9d4a-4376-90f5-ea5184cb12bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:10.211628 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45a3e9-9d4a-4376-90f5-ea5184cb12bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:10.211895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45a3e9-9d4a-4376-90f5-ea5184cb12bf admin admin] PATH_INFO: `/v3/users/c8e0eef3e17f4f27b5048316a7a74dc2` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:10.213785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45a3e9-9d4a-4376-90f5-ea5184cb12bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8e0eef3e17f4f27b5048316a7a74dc2)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:10.216901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45a3e9-9d4a-4376-90f5-ea5184cb12bf admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:10.274515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b891f821-d240-44e6-8a7d-7543a32c37fa None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:10.278968 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-ec45a3e9-9d4a-4376-90f5-ea5184cb12bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8e0eef3e17f4f27b5048316a7a74dc2'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:10.279445 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3aca43-7d41-4647-9dcd-3e6f1c89c3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:10.280073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aca43-7d41-4647-9dcd-3e6f1c89c3c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:10.280352 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aca43-7d41-4647-9dcd-3e6f1c89c3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:10.281207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3aca43-7d41-4647-9dcd-3e6f1c89c3c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:10.281935 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3aca43-7d41-4647-9dcd-3e6f1c89c3c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:10.283528 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3aca43-7d41-4647-9dcd-3e6f1c89c3c7 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:10.288117 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-ec45a3e9-9d4a-4376-90f5-ea5184cb12bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8e0eef3e17f4f27b5048316a7a74dc2'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:10.289123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-ec45a3e9-9d4a-4376-90f5-ea5184cb12bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8e0eef3e17f4f27b5048316a7a74dc2'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:10.294561 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1316/2640] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:10.295444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3aca43-7d41-4647-9dcd-3e6f1c89c3c7 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:10.308377 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-ec45a3e9-9d4a-4376-90f5-ea5184cb12bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8e0eef3e17f4f27b5048316a7a74dc2'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:10.309384 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-ec45a3e9-9d4a-4376-90f5-ea5184cb12bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8e0eef3e17f4f27b5048316a7a74dc2'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:10.330566 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1325/2641] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:24:10 2022] DELETE /identity/v3/users/c8e0eef3e17f4f27b5048316a7a74dc2 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:10.331602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45a3e9-9d4a-4376-90f5-ea5184cb12bf admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:10.336291 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-ec45a3e9-9d4a-4376-90f5-ea5184cb12bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:10.366627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3aca43-7d41-4647-9dcd-3e6f1c89c3c7 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:10.370261 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:10.394827 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:10.414577 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45a3e9-9d4a-4376-90f5-ea5184cb12bf admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:10.420101 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dafbda-647f-4395-9d57-c4cb64d5ada7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:10.420917 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dafbda-647f-4395-9d57-c4cb64d5ada7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:10.421677 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dafbda-647f-4395-9d57-c4cb64d5ada7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:10.421963 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dafbda-647f-4395-9d57-c4cb64d5ada7 admin admin] PATH_INFO: `/v3/projects/72c769241feb4f20a536bf1cb3b8a13b/users/46053624901c4b55acb1089e9dbbb5ff/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:10.424173 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dafbda-647f-4395-9d57-c4cb64d5ada7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72c769241feb4f20a536bf1cb3b8a13b, user_id=46053624901c4b55acb1089e9dbbb5ff, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:10.426276 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d1b3c-b2ac-4340-9bb1-72b63754a28f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:10.427246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d1b3c-b2ac-4340-9bb1-72b63754a28f service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:10.427246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d1b3c-b2ac-4340-9bb1-72b63754a28f service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:10.427506 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dafbda-647f-4395-9d57-c4cb64d5ada7 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:10.427612 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d1b3c-b2ac-4340-9bb1-72b63754a28f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:10.429869 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-633d1b3c-b2ac-4340-9bb1-72b63754a28f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:10.446584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1326/2642] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:10 2022] PUT /identity/v3/projects/72c769241feb4f20a536bf1cb3b8a13b/users/46053624901c4b55acb1089e9dbbb5ff/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:10.446584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a40556-3e8e-48c0-85a8-a276c56acdf6 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:10.446584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a40556-3e8e-48c0-85a8-a276c56acdf6 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:10.446584 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a40556-3e8e-48c0-85a8-a276c56acdf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:10.467013 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-02a40556-3e8e-48c0-85a8-a276c56acdf6 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:10.468599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-02a40556-3e8e-48c0-85a8-a276c56acdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:10.553335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1327/2643] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:10.565249 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a40556-3e8e-48c0-85a8-a276c56acdf6 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:10.567012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-02a40556-3e8e-48c0-85a8-a276c56acdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:10.594845 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d1b3c-b2ac-4340-9bb1-72b63754a28f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:10.596314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d1b3c-b2ac-4340-9bb1-72b63754a28f service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:10.607326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1317/2644] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:10.669525 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a40556-3e8e-48c0-85a8-a276c56acdf6 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:10.682391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b931f-b44e-45a7-b657-6fb4b95b35e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:10.686703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b931f-b44e-45a7-b657-6fb4b95b35e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:10.689211 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b931f-b44e-45a7-b657-6fb4b95b35e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:10.690421 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b931f-b44e-45a7-b657-6fb4b95b35e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:10.726009 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b931f-b44e-45a7-b657-6fb4b95b35e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:10.730630 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b931f-b44e-45a7-b657-6fb4b95b35e7 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:10.755788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1328/2645] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:10.778221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d1b3c-b2ac-4340-9bb1-72b63754a28f service cinder] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:10.780118 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-633d1b3c-b2ac-4340-9bb1-72b63754a28f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:10.793991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d1b3c-b2ac-4340-9bb1-72b63754a28f service cinder] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:10.891536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905e2d0-41b1-4ced-a03d-47f3ad4cadfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:10.892423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905e2d0-41b1-4ced-a03d-47f3ad4cadfa service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:10.893267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905e2d0-41b1-4ced-a03d-47f3ad4cadfa service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:10.893749 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905e2d0-41b1-4ced-a03d-47f3ad4cadfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:10.899081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-c905e2d0-41b1-4ced-a03d-47f3ad4cadfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:10.993833 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905e2d0-41b1-4ced-a03d-47f3ad4cadfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:10.995490 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905e2d0-41b1-4ced-a03d-47f3ad4cadfa service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:11.011256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1318/2646] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:11.308692 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b931f-b44e-45a7-b657-6fb4b95b35e7 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:11.310259 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-339b931f-b44e-45a7-b657-6fb4b95b35e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:11.331165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b931f-b44e-45a7-b657-6fb4b95b35e7 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:11.337144 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c193a0a8-e78d-4841-bd0e-0e5909f5cd41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:11.337865 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193a0a8-e78d-4841-bd0e-0e5909f5cd41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:11.338224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193a0a8-e78d-4841-bd0e-0e5909f5cd41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:11.338522 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193a0a8-e78d-4841-bd0e-0e5909f5cd41 admin admin] PATH_INFO: `/v3/projects/6e464a33c8244695b19722e2f27020fd` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:11.339828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193a0a8-e78d-4841-bd0e-0e5909f5cd41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e464a33c8244695b19722e2f27020fd)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:11.341892 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c193a0a8-e78d-4841-bd0e-0e5909f5cd41 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:11.420302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-c193a0a8-e78d-4841-bd0e-0e5909f5cd41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e464a33c8244695b19722e2f27020fd', 'request_id': 'req-c193a0a8-e78d-4841-bd0e-0e5909f5cd41'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:11.433436 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-c193a0a8-e78d-4841-bd0e-0e5909f5cd41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e464a33c8244695b19722e2f27020fd', 'request_id': 'req-c193a0a8-e78d-4841-bd0e-0e5909f5cd41'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:11.441661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:11.444413 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-c193a0a8-e78d-4841-bd0e-0e5909f5cd41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e464a33c8244695b19722e2f27020fd', 'request_id': 'req-c193a0a8-e78d-4841-bd0e-0e5909f5cd41'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:11.460258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-c193a0a8-e78d-4841-bd0e-0e5909f5cd41 admin admin] The token cache is being invalidate because project 6e464a33c8244695b19722e2f27020fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:24:11.460496 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-c193a0a8-e78d-4841-bd0e-0e5909f5cd41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:11.461194 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-c193a0a8-e78d-4841-bd0e-0e5909f5cd41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:11.462804 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1329/2647] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:24:11 2022] DELETE /identity/v3/projects/6e464a33c8244695b19722e2f27020fd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:11.469000 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b98a5-36bb-46c8-97e7-217bca204813 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:11.469768 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b98a5-36bb-46c8-97e7-217bca204813 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:11.469850 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11b98a5-36bb-46c8-97e7-217bca204813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:11.492951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-c11b98a5-36bb-46c8-97e7-217bca204813 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:11.494262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-c11b98a5-36bb-46c8-97e7-217bca204813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:11.523652 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:11.570605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1319/2648] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:11.579502 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa058ba-2bf4-49bb-b159-38967f5a20c7 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:11.579781 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa058ba-2bf4-49bb-b159-38967f5a20c7 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:11.580058 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa058ba-2bf4-49bb-b159-38967f5a20c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:11.601226 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-faa058ba-2bf4-49bb-b159-38967f5a20c7 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:11.602582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-faa058ba-2bf4-49bb-b159-38967f5a20c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:11.613981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1330/2649] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:11.619806 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b98a5-36bb-46c8-97e7-217bca204813 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:11.638997 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11b98a5-36bb-46c8-97e7-217bca204813 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:11.645211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36274ebf-5912-4e29-b805-8a714a87d060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:11.645903 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36274ebf-5912-4e29-b805-8a714a87d060 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:11.646185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36274ebf-5912-4e29-b805-8a714a87d060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:11.646453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36274ebf-5912-4e29-b805-8a714a87d060 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:11.647152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36274ebf-5912-4e29-b805-8a714a87d060 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:24:11.647454 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36274ebf-5912-4e29-b805-8a714a87d060 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:11.649713 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36274ebf-5912-4e29-b805-8a714a87d060 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:11.664955 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1320/2650] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:24:11.668423 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa058ba-2bf4-49bb-b159-38967f5a20c7 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:11.678495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa058ba-2bf4-49bb-b159-38967f5a20c7 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:11.683205 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fa213-8220-4524-871b-78f810827a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:11.683888 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fa213-8220-4524-871b-78f810827a74 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:11.684230 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fa213-8220-4524-871b-78f810827a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:11.684567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7fa213-8220-4524-871b-78f810827a74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:11.685369 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7fa213-8220-4524-871b-78f810827a74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:11.687879 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7fa213-8220-4524-871b-78f810827a74 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:11.722221 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1331/2651] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:24:11 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:11.735414 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36274ebf-5912-4e29-b805-8a714a87d060 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:11.746259 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36274ebf-5912-4e29-b805-8a714a87d060 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:11.821550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008f228-0045-4dce-8315-53e40fe72ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:11.822314 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008f228-0045-4dce-8315-53e40fe72ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:11.822665 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008f228-0045-4dce-8315-53e40fe72ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:11.823007 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008f228-0045-4dce-8315-53e40fe72ff6 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:11.823780 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008f228-0045-4dce-8315-53e40fe72ff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:11.826477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008f228-0045-4dce-8315-53e40fe72ff6 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:11.853378 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1321/2652] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:24:11 2022] POST /identity/v3/users => generated 286 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:11.857962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fa213-8220-4524-871b-78f810827a74 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:11.870153 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7fa213-8220-4524-871b-78f810827a74 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:11.876427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52b54b-5d3a-4643-80e0-ea972ce1c8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:11.876427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52b54b-5d3a-4643-80e0-ea972ce1c8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:11.876849 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52b54b-5d3a-4643-80e0-ea972ce1c8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:11.877008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52b54b-5d3a-4643-80e0-ea972ce1c8d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:11.879088 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52b54b-5d3a-4643-80e0-ea972ce1c8d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:11.879358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52b54b-5d3a-4643-80e0-ea972ce1c8d1 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:11.893462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1332/2653] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:11.894641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008f228-0045-4dce-8315-53e40fe72ff6 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:11.906551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008f228-0045-4dce-8315-53e40fe72ff6 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:11.912755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ad452-570a-48b2-81b3-deedb2d0b8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:11.912925 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ad452-570a-48b2-81b3-deedb2d0b8df admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:11.915489 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ad452-570a-48b2-81b3-deedb2d0b8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:11.915874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ad452-570a-48b2-81b3-deedb2d0b8df admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:11.916729 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ad452-570a-48b2-81b3-deedb2d0b8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:11.919165 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ad452-570a-48b2-81b3-deedb2d0b8df admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:11.933724 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1322/2654] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:11.934891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52b54b-5d3a-4643-80e0-ea972ce1c8d1 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:11.949709 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52b54b-5d3a-4643-80e0-ea972ce1c8d1 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:11.956119 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39246c2a-3392-42c9-b41b-dd977bbed6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:11.956910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39246c2a-3392-42c9-b41b-dd977bbed6a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:11.957186 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39246c2a-3392-42c9-b41b-dd977bbed6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:11.957517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39246c2a-3392-42c9-b41b-dd977bbed6a9 admin admin] PATH_INFO: `/v3/projects/f3805415af3d49709b95e2b6105f370c/users/32292f5335994ffa9dd566f4e8af1c74/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:11.978611 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39246c2a-3392-42c9-b41b-dd977bbed6a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3805415af3d49709b95e2b6105f370c, user_id=32292f5335994ffa9dd566f4e8af1c74, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:11.981683 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39246c2a-3392-42c9-b41b-dd977bbed6a9 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:12.001432 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1333/2655] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:11 2022] PUT /identity/v3/projects/f3805415af3d49709b95e2b6105f370c/users/32292f5335994ffa9dd566f4e8af1c74/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:12.020296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8566a81-f4d1-4d1d-9697-57a1abf15549 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:12.020644 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8566a81-f4d1-4d1d-9697-57a1abf15549 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:12.020921 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8566a81-f4d1-4d1d-9697-57a1abf15549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:12.063108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-d8566a81-f4d1-4d1d-9697-57a1abf15549 None None] MFA Rules not processed for user `32292f5335994ffa9dd566f4e8af1c74`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:12.064319 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d8566a81-f4d1-4d1d-9697-57a1abf15549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:12.146726 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1323/2656] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:12.151622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39246c2a-3392-42c9-b41b-dd977bbed6a9 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:12.223057 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39246c2a-3392-42c9-b41b-dd977bbed6a9 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:12.229579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd24152-d28f-4fc0-9f29-de36639816ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:12.230220 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd24152-d28f-4fc0-9f29-de36639816ac admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:12.230504 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd24152-d28f-4fc0-9f29-de36639816ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:12.230775 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd24152-d28f-4fc0-9f29-de36639816ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:12.231444 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd24152-d28f-4fc0-9f29-de36639816ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:12.233363 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd24152-d28f-4fc0-9f29-de36639816ac admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:12.268660 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1334/2657] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:24:12 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:12.271544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8566a81-f4d1-4d1d-9697-57a1abf15549 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:12.291394 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8566a81-f4d1-4d1d-9697-57a1abf15549 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:12.336604 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:12.405011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa76272-d215-4634-82a0-655b2fa19cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:12.405723 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa76272-d215-4634-82a0-655b2fa19cb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:12.406153 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa76272-d215-4634-82a0-655b2fa19cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:12.406803 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa76272-d215-4634-82a0-655b2fa19cb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:12.407504 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa76272-d215-4634-82a0-655b2fa19cb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:12.410255 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa76272-d215-4634-82a0-655b2fa19cb3 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:12.451731 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1324/2658] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:24:12 2022] POST /identity/v3/users => generated 294 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:24:12.452354 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd24152-d28f-4fc0-9f29-de36639816ac admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:12.472544 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd24152-d28f-4fc0-9f29-de36639816ac admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:12.478649 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e431a7-b9bc-4701-a772-c9e8da4e69eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:12.479287 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e431a7-b9bc-4701-a772-c9e8da4e69eb admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:12.479566 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e431a7-b9bc-4701-a772-c9e8da4e69eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:12.479844 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e431a7-b9bc-4701-a772-c9e8da4e69eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:12.480891 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e431a7-b9bc-4701-a772-c9e8da4e69eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:12.482957 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e431a7-b9bc-4701-a772-c9e8da4e69eb admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:12.495605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1335/2659] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:24:12.500778 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa76272-d215-4634-82a0-655b2fa19cb3 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:12.512646 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa76272-d215-4634-82a0-655b2fa19cb3 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:12.519012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f676e4e-31c2-47b7-be33-58ad035f7802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:12.520399 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f676e4e-31c2-47b7-be33-58ad035f7802 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:12.520742 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f676e4e-31c2-47b7-be33-58ad035f7802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:12.521041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f676e4e-31c2-47b7-be33-58ad035f7802 admin admin] PATH_INFO: `/v3/projects/eea93f45f88847cb81f5902c384a728c/users/758d9817c513412e8d0d10c16fb29396/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:12.540110 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f676e4e-31c2-47b7-be33-58ad035f7802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea93f45f88847cb81f5902c384a728c, user_id=758d9817c513412e8d0d10c16fb29396, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:12.542839 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f676e4e-31c2-47b7-be33-58ad035f7802 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:12.573379 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1325/2660] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:12 2022] PUT /identity/v3/projects/eea93f45f88847cb81f5902c384a728c/users/758d9817c513412e8d0d10c16fb29396/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:12.578291 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e431a7-b9bc-4701-a772-c9e8da4e69eb admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:12.590026 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e431a7-b9bc-4701-a772-c9e8da4e69eb admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:12.685752 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab07384-5eec-4c98-8a85-43704dd4e838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:12.686856 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab07384-5eec-4c98-8a85-43704dd4e838 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:12.686856 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab07384-5eec-4c98-8a85-43704dd4e838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:12.687182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab07384-5eec-4c98-8a85-43704dd4e838 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:12.687661 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab07384-5eec-4c98-8a85-43704dd4e838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:12.689495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab07384-5eec-4c98-8a85-43704dd4e838 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:12.691137 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:12.722406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1336/2661] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:12.731459 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f676e4e-31c2-47b7-be33-58ad035f7802 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:12.755176 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f676e4e-31c2-47b7-be33-58ad035f7802 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:12.764278 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3927760-ba74-42e8-862c-3044ff1ad5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:12.766292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3927760-ba74-42e8-862c-3044ff1ad5c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:12.767016 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3927760-ba74-42e8-862c-3044ff1ad5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:12.767424 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3927760-ba74-42e8-862c-3044ff1ad5c8 admin admin] PATH_INFO: `/v3/projects/eea93f45f88847cb81f5902c384a728c/users/758d9817c513412e8d0d10c16fb29396/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:12.770837 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3927760-ba74-42e8-862c-3044ff1ad5c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eea93f45f88847cb81f5902c384a728c, user_id=758d9817c513412e8d0d10c16fb29396, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:12.776021 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3927760-ba74-42e8-862c-3044ff1ad5c8 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:12.802483 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1326/2662] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:12 2022] PUT /identity/v3/projects/eea93f45f88847cb81f5902c384a728c/users/758d9817c513412e8d0d10c16fb29396/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:12.812572 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e30367-fc8e-4dff-bde4-a9b26ecfe327 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:12.812963 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e30367-fc8e-4dff-bde4-a9b26ecfe327 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:12.813165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e30367-fc8e-4dff-bde4-a9b26ecfe327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:12.866909 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-e8e30367-fc8e-4dff-bde4-a9b26ecfe327 None None] MFA Rules not processed for user `758d9817c513412e8d0d10c16fb29396`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:12.868581 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e8e30367-fc8e-4dff-bde4-a9b26ecfe327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:12.982742 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1337/2663] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:24:13.001981 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3927760-ba74-42e8-862c-3044ff1ad5c8 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:13.004961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d3927760-ba74-42e8-862c-3044ff1ad5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:13.131072 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3927760-ba74-42e8-862c-3044ff1ad5c8 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:13.136557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82683bc-4389-47ba-99d6-5e808381fa1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:13.137709 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82683bc-4389-47ba-99d6-5e808381fa1b service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:13.138217 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82683bc-4389-47ba-99d6-5e808381fa1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:13.138716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82683bc-4389-47ba-99d6-5e808381fa1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:13.227006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82683bc-4389-47ba-99d6-5e808381fa1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:13.228701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82683bc-4389-47ba-99d6-5e808381fa1b service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:13.240292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1327/2664] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:16.348565 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357aea2-9406-46d9-aa8d-515a0bc64042 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:16.349181 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357aea2-9406-46d9-aa8d-515a0bc64042 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:16.349181 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357aea2-9406-46d9-aa8d-515a0bc64042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:16.381825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-a357aea2-9406-46d9-aa8d-515a0bc64042 None None] MFA Rules not processed for user `46053624901c4b55acb1089e9dbbb5ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:16.383060 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a357aea2-9406-46d9-aa8d-515a0bc64042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:16.429325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82683bc-4389-47ba-99d6-5e808381fa1b service cinder] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:16.464292 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1338/2665] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:16.474591 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:16.477580 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357aea2-9406-46d9-aa8d-515a0bc64042 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:16.479451 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a357aea2-9406-46d9-aa8d-515a0bc64042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:16.512171 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82683bc-4389-47ba-99d6-5e808381fa1b service cinder] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:16.517163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b072e60-3454-428f-a33a-71c28bca52d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:16.517854 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b072e60-3454-428f-a33a-71c28bca52d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:16.518246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b072e60-3454-428f-a33a-71c28bca52d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:16.518398 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b072e60-3454-428f-a33a-71c28bca52d9 admin admin] PATH_INFO: `/v3/users/32292f5335994ffa9dd566f4e8af1c74` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:16.519579 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b072e60-3454-428f-a33a-71c28bca52d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32292f5335994ffa9dd566f4e8af1c74)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:16.521561 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b072e60-3454-428f-a33a-71c28bca52d9 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:16.545341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357aea2-9406-46d9-aa8d-515a0bc64042 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:16.550371 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec7058-59bb-4362-9b90-3c6080f669f1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14ff31d52ecf4c7d93a65bbfca7285c0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:16.551069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec7058-59bb-4362-9b90-3c6080f669f1 service glance] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:16.551418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec7058-59bb-4362-9b90-3c6080f669f1 service glance] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:16.551715 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec7058-59bb-4362-9b90-3c6080f669f1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:16.554008 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-aeec7058-59bb-4362-9b90-3c6080f669f1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:16.582184 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:16.586668 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1b072e60-3454-428f-a33a-71c28bca52d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32292f5335994ffa9dd566f4e8af1c74'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:16.587452 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1b072e60-3454-428f-a33a-71c28bca52d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32292f5335994ffa9dd566f4e8af1c74'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:16.589064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1b072e60-3454-428f-a33a-71c28bca52d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32292f5335994ffa9dd566f4e8af1c74'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:16.607864 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1b072e60-3454-428f-a33a-71c28bca52d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32292f5335994ffa9dd566f4e8af1c74'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:16.619032 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-1b072e60-3454-428f-a33a-71c28bca52d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32292f5335994ffa9dd566f4e8af1c74'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:16.628864 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec7058-59bb-4362-9b90-3c6080f669f1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:16.630910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec7058-59bb-4362-9b90-3c6080f669f1 service glance] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:16.635798 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1328/2666] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:24:16 2022] DELETE /identity/v3/users/32292f5335994ffa9dd566f4e8af1c74 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:24:16.641808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1339/2667] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:24:16.642431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a137f94-b7f1-47db-8593-4f889b5e05f7 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:16.642701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a137f94-b7f1-47db-8593-4f889b5e05f7 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:16.642999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a137f94-b7f1-47db-8593-4f889b5e05f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:16.669184 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-9a137f94-b7f1-47db-8593-4f889b5e05f7 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:16.670448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9a137f94-b7f1-47db-8593-4f889b5e05f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:16.783520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d572d76-768f-4745-8d20-bb34ee8cdda9 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:16.784110 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d572d76-768f-4745-8d20-bb34ee8cdda9 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:16.784362 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d572d76-768f-4745-8d20-bb34ee8cdda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:16.805739 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-7d572d76-768f-4745-8d20-bb34ee8cdda9 None None] MFA Rules not processed for user `e54e713b52e84393b018807f8153922e`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:16.806975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7d572d76-768f-4745-8d20-bb34ee8cdda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:16.869139 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1329/2668] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:16.878999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a137f94-b7f1-47db-8593-4f889b5e05f7 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:16.880439 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9a137f94-b7f1-47db-8593-4f889b5e05f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:16.884226 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1340/2669] 10.222.0.45 () {58 vars in 1009 bytes} [Thu Aug 4 02:24:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:24:16.896841 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d572d76-768f-4745-8d20-bb34ee8cdda9 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:16.898234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7d572d76-768f-4745-8d20-bb34ee8cdda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:16.940315 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a137f94-b7f1-47db-8593-4f889b5e05f7 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:16.944087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38b4f4-7102-4c94-b23d-2d61a84cfca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:16.944731 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38b4f4-7102-4c94-b23d-2d61a84cfca9 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:16.945066 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38b4f4-7102-4c94-b23d-2d61a84cfca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:16.945349 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38b4f4-7102-4c94-b23d-2d61a84cfca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:16.952606 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:16.955177 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38b4f4-7102-4c94-b23d-2d61a84cfca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:16.957083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df38b4f4-7102-4c94-b23d-2d61a84cfca9 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:16.969313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1330/2670] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:16.975640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d572d76-768f-4745-8d20-bb34ee8cdda9 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:16.981071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e603b5e-5c96-4249-a4b2-18f9899e2341 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '155a7276efab426cb34175683c475f33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:16.981697 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e603b5e-5c96-4249-a4b2-18f9899e2341 service swift] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:16.981975 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e603b5e-5c96-4249-a4b2-18f9899e2341 service swift] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:16.982247 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e603b5e-5c96-4249-a4b2-18f9899e2341 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:16.991460 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e603b5e-5c96-4249-a4b2-18f9899e2341 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:16.993062 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e603b5e-5c96-4249-a4b2-18f9899e2341 service swift] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:17.001328 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1341/2671] 10.222.0.45 () {60 vars in 1384 bytes} [Thu Aug 4 02:24:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 02:24:17.342163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38b4f4-7102-4c94-b23d-2d61a84cfca9 service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:17.346115 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-df38b4f4-7102-4c94-b23d-2d61a84cfca9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:17.370328 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df38b4f4-7102-4c94-b23d-2d61a84cfca9 service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:17.388166 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ec514-2117-4222-bb0d-8a66415b7c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:17.388923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ec514-2117-4222-bb0d-8a66415b7c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:17.389190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ec514-2117-4222-bb0d-8a66415b7c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:17.389461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ec514-2117-4222-bb0d-8a66415b7c2a admin admin] PATH_INFO: `/v3/projects/f3805415af3d49709b95e2b6105f370c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:17.391177 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ec514-2117-4222-bb0d-8a66415b7c2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3805415af3d49709b95e2b6105f370c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:17.393191 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ec514-2117-4222-bb0d-8a66415b7c2a admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:17.458776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-418ec514-2117-4222-bb0d-8a66415b7c2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3805415af3d49709b95e2b6105f370c', 'request_id': 'req-418ec514-2117-4222-bb0d-8a66415b7c2a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:17.472951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-418ec514-2117-4222-bb0d-8a66415b7c2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3805415af3d49709b95e2b6105f370c', 'request_id': 'req-418ec514-2117-4222-bb0d-8a66415b7c2a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:17.487359 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-418ec514-2117-4222-bb0d-8a66415b7c2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3805415af3d49709b95e2b6105f370c', 'request_id': 'req-418ec514-2117-4222-bb0d-8a66415b7c2a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:17.505503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-418ec514-2117-4222-bb0d-8a66415b7c2a admin admin] The token cache is being invalidate because project f3805415af3d49709b95e2b6105f370c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:24:17.505790 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-418ec514-2117-4222-bb0d-8a66415b7c2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:17.506625 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-418ec514-2117-4222-bb0d-8a66415b7c2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:17.511208 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1331/2672] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:24:17 2022] DELETE /identity/v3/projects/f3805415af3d49709b95e2b6105f370c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:17.513035 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e603b5e-5c96-4249-a4b2-18f9899e2341 service swift] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:17.526148 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e603b5e-5c96-4249-a4b2-18f9899e2341 service swift] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:17.600580 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:17.614179 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c12943-2b09-4f05-a944-7f3bf74d7ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:17.614966 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c12943-2b09-4f05-a944-7f3bf74d7ded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:17.615298 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c12943-2b09-4f05-a944-7f3bf74d7ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:17.615638 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c12943-2b09-4f05-a944-7f3bf74d7ded admin admin] PATH_INFO: `/v3/users/758d9817c513412e8d0d10c16fb29396` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:17.617341 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c12943-2b09-4f05-a944-7f3bf74d7ded admin admin] RBAC: Authorizing `identity:delete_user(user_id=758d9817c513412e8d0d10c16fb29396)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:17.620024 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c12943-2b09-4f05-a944-7f3bf74d7ded admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:17.705862 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-43c12943-2b09-4f05-a944-7f3bf74d7ded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '758d9817c513412e8d0d10c16fb29396'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:17.716518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-43c12943-2b09-4f05-a944-7f3bf74d7ded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '758d9817c513412e8d0d10c16fb29396'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:17.717308 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-43c12943-2b09-4f05-a944-7f3bf74d7ded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '758d9817c513412e8d0d10c16fb29396'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:17.745904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-43c12943-2b09-4f05-a944-7f3bf74d7ded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '758d9817c513412e8d0d10c16fb29396'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:17.747683 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-43c12943-2b09-4f05-a944-7f3bf74d7ded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '758d9817c513412e8d0d10c16fb29396'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:17.764664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1342/2673] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:24:17 2022] DELETE /identity/v3/users/758d9817c513412e8d0d10c16fb29396 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:18.217653 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ec514-2117-4222-bb0d-8a66415b7c2a admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:18.223409 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-418ec514-2117-4222-bb0d-8a66415b7c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:18.250782 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ec514-2117-4222-bb0d-8a66415b7c2a admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:18.333150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b6313-9a58-4322-a140-d9c165f86d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:18.333823 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b6313-9a58-4322-a140-d9c165f86d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:18.334091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b6313-9a58-4322-a140-d9c165f86d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:18.334370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b6313-9a58-4322-a140-d9c165f86d8f admin admin] PATH_INFO: `/v3/projects/eea93f45f88847cb81f5902c384a728c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:18.335682 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b6313-9a58-4322-a140-d9c165f86d8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=eea93f45f88847cb81f5902c384a728c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:18.337641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b6313-9a58-4322-a140-d9c165f86d8f admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:18.448497 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4f9b6313-9a58-4322-a140-d9c165f86d8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eea93f45f88847cb81f5902c384a728c', 'request_id': 'req-4f9b6313-9a58-4322-a140-d9c165f86d8f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:18.461703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4f9b6313-9a58-4322-a140-d9c165f86d8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eea93f45f88847cb81f5902c384a728c', 'request_id': 'req-4f9b6313-9a58-4322-a140-d9c165f86d8f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:18.473941 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4f9b6313-9a58-4322-a140-d9c165f86d8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eea93f45f88847cb81f5902c384a728c', 'request_id': 'req-4f9b6313-9a58-4322-a140-d9c165f86d8f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:18.489752 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4f9b6313-9a58-4322-a140-d9c165f86d8f admin admin] The token cache is being invalidate because project eea93f45f88847cb81f5902c384a728c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:24:18.490395 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4f9b6313-9a58-4322-a140-d9c165f86d8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:18.491308 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4f9b6313-9a58-4322-a140-d9c165f86d8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:18.495971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1332/2674] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:24:18 2022] DELETE /identity/v3/projects/eea93f45f88847cb81f5902c384a728c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:18.498484 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03ad4f-6f34-43d4-a061-3f27a4df3e25 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:18.498778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03ad4f-6f34-43d4-a061-3f27a4df3e25 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:18.499061 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd03ad4f-6f34-43d4-a061-3f27a4df3e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:18.526098 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-bd03ad4f-6f34-43d4-a061-3f27a4df3e25 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:18.527623 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-bd03ad4f-6f34-43d4-a061-3f27a4df3e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:18.608338 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1343/2675] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:18.616745 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e6ce9-bc42-4abb-8614-1b79fe1d752a None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:18.617292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e6ce9-bc42-4abb-8614-1b79fe1d752a None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:18.617738 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e6ce9-bc42-4abb-8614-1b79fe1d752a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:18.662178 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-718e6ce9-bc42-4abb-8614-1b79fe1d752a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:18.663516 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-718e6ce9-bc42-4abb-8614-1b79fe1d752a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:18.676768 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1333/2676] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:18.681607 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03ad4f-6f34-43d4-a061-3f27a4df3e25 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:18.693328 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd03ad4f-6f34-43d4-a061-3f27a4df3e25 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:18.704351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ab2b5-f6b9-4abe-b68b-2b7768bf808f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:18.705615 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ab2b5-f6b9-4abe-b68b-2b7768bf808f admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:18.706056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ab2b5-f6b9-4abe-b68b-2b7768bf808f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:18.706498 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ab2b5-f6b9-4abe-b68b-2b7768bf808f admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:18.707291 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ab2b5-f6b9-4abe-b68b-2b7768bf808f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:24:18.707677 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ab2b5-f6b9-4abe-b68b-2b7768bf808f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:18.709543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ab2b5-f6b9-4abe-b68b-2b7768bf808f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:18.749995 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e6ce9-bc42-4abb-8614-1b79fe1d752a None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:18.750274 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1344/2677] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:24:18.754906 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:18.774255 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e6ce9-bc42-4abb-8614-1b79fe1d752a None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:18.796200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d6d00-4e87-4507-8a94-785bd436598b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:18.796927 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d6d00-4e87-4507-8a94-785bd436598b admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:18.797138 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d6d00-4e87-4507-8a94-785bd436598b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:18.797431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3d6d00-4e87-4507-8a94-785bd436598b admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:18.798078 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d6d00-4e87-4507-8a94-785bd436598b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:18.801537 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3d6d00-4e87-4507-8a94-785bd436598b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:18.859114 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1334/2678] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:24:18 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:18.864571 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ab2b5-f6b9-4abe-b68b-2b7768bf808f admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:18.877071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ab2b5-f6b9-4abe-b68b-2b7768bf808f admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:19.038815 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd68fb-a896-4f37-bd82-9fbee2fd5282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:19.039994 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd68fb-a896-4f37-bd82-9fbee2fd5282 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:19.040483 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd68fb-a896-4f37-bd82-9fbee2fd5282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:19.041017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd68fb-a896-4f37-bd82-9fbee2fd5282 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:19.043538 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd68fb-a896-4f37-bd82-9fbee2fd5282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:19.050703 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd68fb-a896-4f37-bd82-9fbee2fd5282 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:19.061622 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:19.088259 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1345/2679] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:24:18 2022] POST /identity/v3/users => generated 287 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:19.090024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d6d00-4e87-4507-8a94-785bd436598b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:19.106555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3d6d00-4e87-4507-8a94-785bd436598b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:19.116065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36a4fb-a630-418e-a1d2-9f50ee123f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:19.117152 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36a4fb-a630-418e-a1d2-9f50ee123f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:19.118934 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36a4fb-a630-418e-a1d2-9f50ee123f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:19.119233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36a4fb-a630-418e-a1d2-9f50ee123f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:19.121552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36a4fb-a630-418e-a1d2-9f50ee123f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:19.128858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36a4fb-a630-418e-a1d2-9f50ee123f18 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:19.149262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1335/2680] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:19.156393 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd68fb-a896-4f37-bd82-9fbee2fd5282 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:19.169106 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd68fb-a896-4f37-bd82-9fbee2fd5282 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:19.175809 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b742e91-f661-4650-b224-800d11ec85c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:19.176528 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b742e91-f661-4650-b224-800d11ec85c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:19.176852 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b742e91-f661-4650-b224-800d11ec85c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:19.177130 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b742e91-f661-4650-b224-800d11ec85c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:19.178118 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b742e91-f661-4650-b224-800d11ec85c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:19.182855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b742e91-f661-4650-b224-800d11ec85c9 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:19.192469 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1346/2681] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:19.196989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36a4fb-a630-418e-a1d2-9f50ee123f18 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:19.220068 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36a4fb-a630-418e-a1d2-9f50ee123f18 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:19.227652 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97452358-aaf4-4e7b-a959-b0b55c547786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:19.245477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97452358-aaf4-4e7b-a959-b0b55c547786 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:19.246156 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97452358-aaf4-4e7b-a959-b0b55c547786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:19.249251 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97452358-aaf4-4e7b-a959-b0b55c547786 admin admin] PATH_INFO: `/v3/projects/bad68421030f4e719c3884b91dcca946/users/368be3b4d0134cea9b44a09c2d80c91b/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:19.294594 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97452358-aaf4-4e7b-a959-b0b55c547786 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad68421030f4e719c3884b91dcca946, user_id=368be3b4d0134cea9b44a09c2d80c91b, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:19.300049 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97452358-aaf4-4e7b-a959-b0b55c547786 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:19.324211 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1336/2682] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:19 2022] PUT /identity/v3/projects/bad68421030f4e719c3884b91dcca946/users/368be3b4d0134cea9b44a09c2d80c91b/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:19.335775 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a2d4c-4dd8-40f7-8fb6-f363edae8664 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:19.336168 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a2d4c-4dd8-40f7-8fb6-f363edae8664 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:19.336449 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150a2d4c-4dd8-40f7-8fb6-f363edae8664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:19.394533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-150a2d4c-4dd8-40f7-8fb6-f363edae8664 None None] MFA Rules not processed for user `368be3b4d0134cea9b44a09c2d80c91b`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:19.396289 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-150a2d4c-4dd8-40f7-8fb6-f363edae8664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:19.517319 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1347/2683] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:24:19.529967 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97452358-aaf4-4e7b-a959-b0b55c547786 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:19.532833 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-97452358-aaf4-4e7b-a959-b0b55c547786 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:19.616842 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97452358-aaf4-4e7b-a959-b0b55c547786 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:19.621033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95626c8c-9ba2-49d5-9887-49fca59e8d20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:19.622629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95626c8c-9ba2-49d5-9887-49fca59e8d20 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:19.622629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95626c8c-9ba2-49d5-9887-49fca59e8d20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:19.622629 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95626c8c-9ba2-49d5-9887-49fca59e8d20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:19.634989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95626c8c-9ba2-49d5-9887-49fca59e8d20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:19.636780 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95626c8c-9ba2-49d5-9887-49fca59e8d20 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:19.648041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1337/2684] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:21.215635 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a2d4c-4dd8-40f7-8fb6-f363edae8664 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:21.217135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-150a2d4c-4dd8-40f7-8fb6-f363edae8664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:21.345422 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150a2d4c-4dd8-40f7-8fb6-f363edae8664 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:21.354806 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fb8e5-0845-47dd-94ba-302e03160379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:21.355756 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fb8e5-0845-47dd-94ba-302e03160379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:21.356116 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fb8e5-0845-47dd-94ba-302e03160379 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:21.356642 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03fb8e5-0845-47dd-94ba-302e03160379 admin admin] PATH_INFO: `/v3/users/6e38e40018474d7098023745a1052f74` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:21.358088 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fb8e5-0845-47dd-94ba-302e03160379 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e38e40018474d7098023745a1052f74)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:21.360501 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03fb8e5-0845-47dd-94ba-302e03160379 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:21.473373 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a03fb8e5-0845-47dd-94ba-302e03160379 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e38e40018474d7098023745a1052f74'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:21.482976 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a03fb8e5-0845-47dd-94ba-302e03160379 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e38e40018474d7098023745a1052f74'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:21.483946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a03fb8e5-0845-47dd-94ba-302e03160379 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e38e40018474d7098023745a1052f74'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:21.511981 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:21.514987 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a03fb8e5-0845-47dd-94ba-302e03160379 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e38e40018474d7098023745a1052f74'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:21.518041 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a03fb8e5-0845-47dd-94ba-302e03160379 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e38e40018474d7098023745a1052f74'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:21.545112 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1348/2685] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:24:21 2022] DELETE /identity/v3/users/6e38e40018474d7098023745a1052f74 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:21.550377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df447b-8777-48e2-afcc-a475d827ac7b None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:21.550881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df447b-8777-48e2-afcc-a475d827ac7b None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:21.551002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df447b-8777-48e2-afcc-a475d827ac7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:21.581819 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-61df447b-8777-48e2-afcc-a475d827ac7b None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:21.584209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-61df447b-8777-48e2-afcc-a475d827ac7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:21.714495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1338/2686] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:21.724741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fb8e5-0845-47dd-94ba-302e03160379 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:21.726601 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a03fb8e5-0845-47dd-94ba-302e03160379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:21.818828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03fb8e5-0845-47dd-94ba-302e03160379 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:21.825877 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2aa50b-c5b8-4aba-b5fc-751eeceb2735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:21.826452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2aa50b-c5b8-4aba-b5fc-751eeceb2735 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:21.826735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2aa50b-c5b8-4aba-b5fc-751eeceb2735 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:21.827015 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2aa50b-c5b8-4aba-b5fc-751eeceb2735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:21.838794 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2aa50b-c5b8-4aba-b5fc-751eeceb2735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:21.840928 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2aa50b-c5b8-4aba-b5fc-751eeceb2735 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:21.861055 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1349/2687] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:22.295192 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df447b-8777-48e2-afcc-a475d827ac7b None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:22.297260 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-61df447b-8777-48e2-afcc-a475d827ac7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:22.318442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df447b-8777-48e2-afcc-a475d827ac7b None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:22.334967 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2e16c-582f-4eb1-af45-aeaab471013a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:22.335935 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2e16c-582f-4eb1-af45-aeaab471013a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:22.336419 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2e16c-582f-4eb1-af45-aeaab471013a admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:22.337087 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2e16c-582f-4eb1-af45-aeaab471013a admin admin] PATH_INFO: `/v3/projects/cc6ac4a9576b4e4abf1a23eaa5a2b735` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:22.339904 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2e16c-582f-4eb1-af45-aeaab471013a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc6ac4a9576b4e4abf1a23eaa5a2b735)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:22.342321 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2e16c-582f-4eb1-af45-aeaab471013a admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:22.436346 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:22.441440 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-67b2e16c-582f-4eb1-af45-aeaab471013a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc6ac4a9576b4e4abf1a23eaa5a2b735', 'request_id': 'req-67b2e16c-582f-4eb1-af45-aeaab471013a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:22.456361 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-67b2e16c-582f-4eb1-af45-aeaab471013a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc6ac4a9576b4e4abf1a23eaa5a2b735', 'request_id': 'req-67b2e16c-582f-4eb1-af45-aeaab471013a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:22.470132 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-67b2e16c-582f-4eb1-af45-aeaab471013a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc6ac4a9576b4e4abf1a23eaa5a2b735', 'request_id': 'req-67b2e16c-582f-4eb1-af45-aeaab471013a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:22.487663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-67b2e16c-582f-4eb1-af45-aeaab471013a admin admin] The token cache is being invalidate because project cc6ac4a9576b4e4abf1a23eaa5a2b735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:24:22.488090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-67b2e16c-582f-4eb1-af45-aeaab471013a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:22.490171 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-67b2e16c-582f-4eb1-af45-aeaab471013a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:22.491987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1339/2688] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:24:22 2022] DELETE /identity/v3/projects/cc6ac4a9576b4e4abf1a23eaa5a2b735 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:23.328060 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f90fae-cc14-446a-ad34-b43775619497 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:23.329056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f90fae-cc14-446a-ad34-b43775619497 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:23.329570 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f90fae-cc14-446a-ad34-b43775619497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:23.352502 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-b2f90fae-cc14-446a-ad34-b43775619497 None None] MFA Rules not processed for user `683f8f120df946f59505455a9ab2dc2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:23.354191 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-b2f90fae-cc14-446a-ad34-b43775619497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:23.435843 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1350/2689] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:23.470080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2e16c-582f-4eb1-af45-aeaab471013a admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:23.472170 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-67b2e16c-582f-4eb1-af45-aeaab471013a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:23.600302 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2e16c-582f-4eb1-af45-aeaab471013a admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:23.606472 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7962b-88ac-4c8e-bf6a-08a9cf33a166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:23.607330 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7962b-88ac-4c8e-bf6a-08a9cf33a166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:23.608060 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7962b-88ac-4c8e-bf6a-08a9cf33a166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:23.608525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7962b-88ac-4c8e-bf6a-08a9cf33a166 admin admin] PATH_INFO: `/v3/users/368be3b4d0134cea9b44a09c2d80c91b` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:23.610132 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7962b-88ac-4c8e-bf6a-08a9cf33a166 admin admin] RBAC: Authorizing `identity:delete_user(user_id=368be3b4d0134cea9b44a09c2d80c91b)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:23.612297 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df7962b-88ac-4c8e-bf6a-08a9cf33a166 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:23.718238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-9df7962b-88ac-4c8e-bf6a-08a9cf33a166 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '368be3b4d0134cea9b44a09c2d80c91b'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:23.718945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-9df7962b-88ac-4c8e-bf6a-08a9cf33a166 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '368be3b4d0134cea9b44a09c2d80c91b'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:23.720716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-9df7962b-88ac-4c8e-bf6a-08a9cf33a166 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '368be3b4d0134cea9b44a09c2d80c91b'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:23.755728 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-9df7962b-88ac-4c8e-bf6a-08a9cf33a166 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '368be3b4d0134cea9b44a09c2d80c91b'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:23.768592 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-9df7962b-88ac-4c8e-bf6a-08a9cf33a166 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '368be3b4d0134cea9b44a09c2d80c91b'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:23.797770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1340/2690] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:24:23 2022] DELETE /identity/v3/users/368be3b4d0134cea9b44a09c2d80c91b => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:23.802220 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1430e-1799-4311-8b72-7c6ba0bde991 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:23.802499 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1430e-1799-4311-8b72-7c6ba0bde991 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:23.803017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1430e-1799-4311-8b72-7c6ba0bde991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:23.824605 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-21b1430e-1799-4311-8b72-7c6ba0bde991 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:23.825940 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-21b1430e-1799-4311-8b72-7c6ba0bde991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:23.925278 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1351/2691] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:23.933286 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7962b-88ac-4c8e-bf6a-08a9cf33a166 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:23.936395 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-9df7962b-88ac-4c8e-bf6a-08a9cf33a166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:23.941023 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1430e-1799-4311-8b72-7c6ba0bde991 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:23.946724 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-21b1430e-1799-4311-8b72-7c6ba0bde991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:24.011744 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:24.018349 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7962b-88ac-4c8e-bf6a-08a9cf33a166 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:24.023032 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c77d1-11e5-4ae4-9a48-f7cf36cc53b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:24.023669 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c77d1-11e5-4ae4-9a48-f7cf36cc53b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:24.023953 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c77d1-11e5-4ae4-9a48-f7cf36cc53b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:24.024230 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c77d1-11e5-4ae4-9a48-f7cf36cc53b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:24.031116 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:24.035613 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1430e-1799-4311-8b72-7c6ba0bde991 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:24.039621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c77d1-11e5-4ae4-9a48-f7cf36cc53b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:24.041179 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c77d1-11e5-4ae4-9a48-f7cf36cc53b9 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:24.047632 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac42fe-6ea9-42c4-a25b-268837ad57cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:24.048272 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac42fe-6ea9-42c4-a25b-268837ad57cf service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:24.048554 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac42fe-6ea9-42c4-a25b-268837ad57cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:24.048862 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac42fe-6ea9-42c4-a25b-268837ad57cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:24.052430 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-95ac42fe-6ea9-42c4-a25b-268837ad57cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:24.062520 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1341/2692] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:24:24.075434 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac42fe-6ea9-42c4-a25b-268837ad57cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:24.077094 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac42fe-6ea9-42c4-a25b-268837ad57cf service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:24.091567 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1352/2693] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:24.459979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c77d1-11e5-4ae4-9a48-f7cf36cc53b9 service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:24.461814 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d90c77d1-11e5-4ae4-9a48-f7cf36cc53b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:24.491462 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c77d1-11e5-4ae4-9a48-f7cf36cc53b9 service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:24.502925 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6f8b6-d64b-47d5-b62a-867a8666d19a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:24.503769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6f8b6-d64b-47d5-b62a-867a8666d19a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:24.504208 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6f8b6-d64b-47d5-b62a-867a8666d19a admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:24.504826 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6f8b6-d64b-47d5-b62a-867a8666d19a admin admin] PATH_INFO: `/v3/projects/bad68421030f4e719c3884b91dcca946` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:24.506193 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6f8b6-d64b-47d5-b62a-867a8666d19a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bad68421030f4e719c3884b91dcca946)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:24.508590 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6f8b6-d64b-47d5-b62a-867a8666d19a admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:24.605955 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-63a6f8b6-d64b-47d5-b62a-867a8666d19a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bad68421030f4e719c3884b91dcca946', 'request_id': 'req-63a6f8b6-d64b-47d5-b62a-867a8666d19a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:24.618936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-63a6f8b6-d64b-47d5-b62a-867a8666d19a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bad68421030f4e719c3884b91dcca946', 'request_id': 'req-63a6f8b6-d64b-47d5-b62a-867a8666d19a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:24.629639 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-63a6f8b6-d64b-47d5-b62a-867a8666d19a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bad68421030f4e719c3884b91dcca946', 'request_id': 'req-63a6f8b6-d64b-47d5-b62a-867a8666d19a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:24.645102 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-63a6f8b6-d64b-47d5-b62a-867a8666d19a admin admin] The token cache is being invalidate because project bad68421030f4e719c3884b91dcca946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:24:24.645743 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-63a6f8b6-d64b-47d5-b62a-867a8666d19a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:24.646692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-63a6f8b6-d64b-47d5-b62a-867a8666d19a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:24.652509 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1342/2694] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:24:24 2022] DELETE /identity/v3/projects/bad68421030f4e719c3884b91dcca946 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:24.653831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e832ac-9233-42c1-8b7e-e922ed85f630 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:24.654360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e832ac-9233-42c1-8b7e-e922ed85f630 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:24.654360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e832ac-9233-42c1-8b7e-e922ed85f630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:24.678010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-30e832ac-9233-42c1-8b7e-e922ed85f630 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:24.679290 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-30e832ac-9233-42c1-8b7e-e922ed85f630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:24.808261 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1353/2695] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:24.811075 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd35dc-7d3b-4d63-bdc1-67ca124d6f18 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:24.812400 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd35dc-7d3b-4d63-bdc1-67ca124d6f18 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:24.813009 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd35dc-7d3b-4d63-bdc1-67ca124d6f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:24.850829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-e0bd35dc-7d3b-4d63-bdc1-67ca124d6f18 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:24.852044 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e0bd35dc-7d3b-4d63-bdc1-67ca124d6f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:24.871410 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1343/2696] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:24.879258 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e832ac-9233-42c1-8b7e-e922ed85f630 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:24.907327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e832ac-9233-42c1-8b7e-e922ed85f630 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:24.928172 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af8e9d-bf12-4111-b72b-17321716f5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:24.928902 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af8e9d-bf12-4111-b72b-17321716f5ea admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:24.929115 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af8e9d-bf12-4111-b72b-17321716f5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:24.929388 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77af8e9d-bf12-4111-b72b-17321716f5ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:24.930095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af8e9d-bf12-4111-b72b-17321716f5ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:24:24.930395 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af8e9d-bf12-4111-b72b-17321716f5ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:24.931988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77af8e9d-bf12-4111-b72b-17321716f5ea admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:24.945699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1354/2697] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:24:24.950196 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd35dc-7d3b-4d63-bdc1-67ca124d6f18 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:24.964449 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd35dc-7d3b-4d63-bdc1-67ca124d6f18 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:24.973207 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d47cd6-60d9-475c-95e1-e521b3c23aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:24.973916 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d47cd6-60d9-475c-95e1-e521b3c23aac admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:24.974306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d47cd6-60d9-475c-95e1-e521b3c23aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:24.974593 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d47cd6-60d9-475c-95e1-e521b3c23aac admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:24.975400 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d47cd6-60d9-475c-95e1-e521b3c23aac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:24.977967 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d47cd6-60d9-475c-95e1-e521b3c23aac admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:25.043677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1344/2698] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:24:24 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:25.045640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af8e9d-bf12-4111-b72b-17321716f5ea admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:25.063625 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77af8e9d-bf12-4111-b72b-17321716f5ea admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:25.151020 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a309352-078f-4052-9c06-5a1f38210bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:25.151779 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a309352-078f-4052-9c06-5a1f38210bdb admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:25.152102 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a309352-078f-4052-9c06-5a1f38210bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:25.152439 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a309352-078f-4052-9c06-5a1f38210bdb admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:25.153212 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a309352-078f-4052-9c06-5a1f38210bdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:25.155261 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a309352-078f-4052-9c06-5a1f38210bdb admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:25.206660 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d47cd6-60d9-475c-95e1-e521b3c23aac admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:25.212387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1355/2699] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:24:25 2022] POST /identity/v3/users => generated 287 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:24:25.228588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d47cd6-60d9-475c-95e1-e521b3c23aac admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:25.243534 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c6ae2-79f4-42b4-aefc-ed232e15580c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:25.244374 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c6ae2-79f4-42b4-aefc-ed232e15580c admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:25.245722 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c6ae2-79f4-42b4-aefc-ed232e15580c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:25.246621 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3c6ae2-79f4-42b4-aefc-ed232e15580c admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:25.247639 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c6ae2-79f4-42b4-aefc-ed232e15580c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:25.251239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3c6ae2-79f4-42b4-aefc-ed232e15580c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:25.269022 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1345/2700] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:25.272633 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a309352-078f-4052-9c06-5a1f38210bdb admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:25.286530 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a309352-078f-4052-9c06-5a1f38210bdb admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:25.298799 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb55d47b-5207-44cd-929f-757132d70002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:25.299855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55d47b-5207-44cd-929f-757132d70002 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:25.300324 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55d47b-5207-44cd-929f-757132d70002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:25.301066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb55d47b-5207-44cd-929f-757132d70002 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:25.302792 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55d47b-5207-44cd-929f-757132d70002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:25.303685 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb55d47b-5207-44cd-929f-757132d70002 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:25.317046 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1356/2701] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:25.321371 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c6ae2-79f4-42b4-aefc-ed232e15580c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:25.337619 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3c6ae2-79f4-42b4-aefc-ed232e15580c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:25.349875 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed978855-273c-4b37-b689-5eaf30881ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:25.350568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed978855-273c-4b37-b689-5eaf30881ef9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:25.350851 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed978855-273c-4b37-b689-5eaf30881ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:25.351204 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed978855-273c-4b37-b689-5eaf30881ef9 admin admin] PATH_INFO: `/v3/projects/f7d4919e934044b7a97dce3ad1cac880/users/b18a6ae879c14eabae7a7a6db120831a/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:25.378494 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed978855-273c-4b37-b689-5eaf30881ef9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d4919e934044b7a97dce3ad1cac880, user_id=b18a6ae879c14eabae7a7a6db120831a, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:25.381846 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed978855-273c-4b37-b689-5eaf30881ef9 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:25.411442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1346/2702] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:25 2022] PUT /identity/v3/projects/f7d4919e934044b7a97dce3ad1cac880/users/b18a6ae879c14eabae7a7a6db120831a/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:25.415655 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8591ae-58a9-4c72-a4cd-f775090f2e7d None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:25.416294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8591ae-58a9-4c72-a4cd-f775090f2e7d None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:25.416797 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8591ae-58a9-4c72-a4cd-f775090f2e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:25.442381 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-7d8591ae-58a9-4c72-a4cd-f775090f2e7d None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:25.443626 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7d8591ae-58a9-4c72-a4cd-f775090f2e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:25.513534 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:25.540161 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1357/2703] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:25.548202 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed978855-273c-4b37-b689-5eaf30881ef9 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:25.550458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ed978855-273c-4b37-b689-5eaf30881ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:25.663309 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed978855-273c-4b37-b689-5eaf30881ef9 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:25.667834 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cec379-1756-44d0-9cfc-cfb99ea913bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:25.668599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cec379-1756-44d0-9cfc-cfb99ea913bf service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:25.668995 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cec379-1756-44d0-9cfc-cfb99ea913bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:25.669341 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cec379-1756-44d0-9cfc-cfb99ea913bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:25.682588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cec379-1756-44d0-9cfc-cfb99ea913bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:25.684029 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cec379-1756-44d0-9cfc-cfb99ea913bf service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:25.697702 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1347/2704] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:26.787325 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bee3a-0dbc-49a9-ba67-5d1eed0f845a None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:26.787908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bee3a-0dbc-49a9-ba67-5d1eed0f845a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:26.789406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bee3a-0dbc-49a9-ba67-5d1eed0f845a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:26.885598 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-a38bee3a-0dbc-49a9-ba67-5d1eed0f845a None None] MFA Rules not processed for user `b18a6ae879c14eabae7a7a6db120831a`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:26.888964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a38bee3a-0dbc-49a9-ba67-5d1eed0f845a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:27.029833 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1358/2705] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:27.041312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cec379-1756-44d0-9cfc-cfb99ea913bf service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:27.043729 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d9cec379-1756-44d0-9cfc-cfb99ea913bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:27.089041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:27.174772 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cec379-1756-44d0-9cfc-cfb99ea913bf service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:27.186006 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce35d7-bb03-4370-93b2-8552aae85f42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:27.187542 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce35d7-bb03-4370-93b2-8552aae85f42 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:27.187991 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce35d7-bb03-4370-93b2-8552aae85f42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:27.188461 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce35d7-bb03-4370-93b2-8552aae85f42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:27.211347 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce35d7-bb03-4370-93b2-8552aae85f42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:27.212826 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce35d7-bb03-4370-93b2-8552aae85f42 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:27.233234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1348/2706] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:28.833307 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29ba72-0dfc-4235-9656-447b4e29ff5b None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:28.834213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29ba72-0dfc-4235-9656-447b4e29ff5b None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:28.835307 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29ba72-0dfc-4235-9656-447b4e29ff5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:28.857465 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-9b29ba72-0dfc-4235-9656-447b4e29ff5b None None] MFA Rules not processed for user `b18a6ae879c14eabae7a7a6db120831a`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:28.858723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9b29ba72-0dfc-4235-9656-447b4e29ff5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:28.869870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1359/2707] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:29.913967 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516857a9-7379-4e06-bdb7-42b2395927bf None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:29.915055 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516857a9-7379-4e06-bdb7-42b2395927bf None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:29.915335 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516857a9-7379-4e06-bdb7-42b2395927bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:29.943953 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-516857a9-7379-4e06-bdb7-42b2395927bf None None] MFA Rules not processed for user `fb3a977fea2c499eb915bc059ab01a79`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:29.946083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-516857a9-7379-4e06-bdb7-42b2395927bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:30.012185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1349/2708] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:30.313466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29ba72-0dfc-4235-9656-447b4e29ff5b None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:30.324035 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29ba72-0dfc-4235-9656-447b4e29ff5b None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:30.328920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80258c57-ee60-4f94-82b9-8ea6abcccce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:30.329536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80258c57-ee60-4f94-82b9-8ea6abcccce9 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:30.329809 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80258c57-ee60-4f94-82b9-8ea6abcccce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:30.330260 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80258c57-ee60-4f94-82b9-8ea6abcccce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:30.332373 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-80258c57-ee60-4f94-82b9-8ea6abcccce9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:30.344077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80258c57-ee60-4f94-82b9-8ea6abcccce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:30.345942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80258c57-ee60-4f94-82b9-8ea6abcccce9 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:30.354437 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1360/2709] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:32.025512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dff02-47da-475b-b657-512e89a86786 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:32.026912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dff02-47da-475b-b657-512e89a86786 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:32.027495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dff02-47da-475b-b657-512e89a86786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:32.050677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-121dff02-47da-475b-b657-512e89a86786 None None] MFA Rules not processed for user `e54e713b52e84393b018807f8153922e`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:32.052226 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-121dff02-47da-475b-b657-512e89a86786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:32.153086 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1350/2710] 10.222.0.45 () {58 vars in 1009 bytes} [Thu Aug 4 02:24:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:32.168941 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80258c57-ee60-4f94-82b9-8ea6abcccce9 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:32.171024 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-80258c57-ee60-4f94-82b9-8ea6abcccce9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:32.251672 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80258c57-ee60-4f94-82b9-8ea6abcccce9 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:32.258131 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc1c28f-59ec-440f-891a-b861d3ac7740 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '155a7276efab426cb34175683c475f33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:32.259128 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1c28f-59ec-440f-891a-b861d3ac7740 service swift] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:32.259399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1c28f-59ec-440f-891a-b861d3ac7740 service swift] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:32.259840 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc1c28f-59ec-440f-891a-b861d3ac7740 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:32.277190 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc1c28f-59ec-440f-891a-b861d3ac7740 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:32.279109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc1c28f-59ec-440f-891a-b861d3ac7740 service swift] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:32.288634 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1361/2711] 10.222.0.45 () {60 vars in 1384 bytes} [Thu Aug 4 02:24:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 02:24:42.145224 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=69 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:42.285010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=79 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:48.993047 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dff02-47da-475b-b657-512e89a86786 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:48.994873 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-121dff02-47da-475b-b657-512e89a86786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:49.007321 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dff02-47da-475b-b657-512e89a86786 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:49.010863 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802510d-5098-4e70-ab27-8967c3112697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:49.011676 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802510d-5098-4e70-ab27-8967c3112697 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:49.011951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802510d-5098-4e70-ab27-8967c3112697 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:49.012219 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802510d-5098-4e70-ab27-8967c3112697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:49.013993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-c802510d-5098-4e70-ab27-8967c3112697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:49.080293 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802510d-5098-4e70-ab27-8967c3112697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:49.085034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802510d-5098-4e70-ab27-8967c3112697 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:49.095125 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1351/2712] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:55.956069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc1c28f-59ec-440f-891a-b861d3ac7740 service swift] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:55.958335 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-0dc1c28f-59ec-440f-891a-b861d3ac7740 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:55.973449 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc1c28f-59ec-440f-891a-b861d3ac7740 service swift] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:55.977822 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950035b-d8ca-484e-81d3-60a953345b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:55.978480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950035b-d8ca-484e-81d3-60a953345b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:55.978735 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950035b-d8ca-484e-81d3-60a953345b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:55.979004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950035b-d8ca-484e-81d3-60a953345b99 admin admin] PATH_INFO: `/v3/users/8206a2d308ab42f3ba5908bd635a38e3` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:55.980202 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950035b-d8ca-484e-81d3-60a953345b99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8206a2d308ab42f3ba5908bd635a38e3)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:55.982252 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950035b-d8ca-484e-81d3-60a953345b99 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:56.033384 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6950035b-d8ca-484e-81d3-60a953345b99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8206a2d308ab42f3ba5908bd635a38e3'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:56.040537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6950035b-d8ca-484e-81d3-60a953345b99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8206a2d308ab42f3ba5908bd635a38e3'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:56.041254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6950035b-d8ca-484e-81d3-60a953345b99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8206a2d308ab42f3ba5908bd635a38e3'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:56.054272 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6950035b-d8ca-484e-81d3-60a953345b99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8206a2d308ab42f3ba5908bd635a38e3'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:56.055076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-6950035b-d8ca-484e-81d3-60a953345b99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8206a2d308ab42f3ba5908bd635a38e3'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:56.070187 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1362/2713] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:24:55 2022] DELETE /identity/v3/users/8206a2d308ab42f3ba5908bd635a38e3 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:56.074762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41727e13-8995-4052-99e4-c3c21a53c11d None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:56.075238 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41727e13-8995-4052-99e4-c3c21a53c11d None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:56.075528 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41727e13-8995-4052-99e4-c3c21a53c11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:56.095690 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-41727e13-8995-4052-99e4-c3c21a53c11d None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:56.097221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-41727e13-8995-4052-99e4-c3c21a53c11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:56.156950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1352/2714] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:24:56.167085 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950035b-d8ca-484e-81d3-60a953345b99 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:56.168510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-6950035b-d8ca-484e-81d3-60a953345b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:56.236331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950035b-d8ca-484e-81d3-60a953345b99 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:56.240704 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fd877-11bf-41c1-8678-a83268f8c541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:56.244348 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fd877-11bf-41c1-8678-a83268f8c541 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:56.245021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fd877-11bf-41c1-8678-a83268f8c541 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:56.245588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0fd877-11bf-41c1-8678-a83268f8c541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:56.255984 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0fd877-11bf-41c1-8678-a83268f8c541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:56.257473 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0fd877-11bf-41c1-8678-a83268f8c541 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:56.268255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1363/2715] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:56.549649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41727e13-8995-4052-99e4-c3c21a53c11d None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:56.551204 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-41727e13-8995-4052-99e4-c3c21a53c11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:56.564398 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41727e13-8995-4052-99e4-c3c21a53c11d None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:56.569015 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaace7e-d889-4039-a4b0-bcc9a62668d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:56.569662 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaace7e-d889-4039-a4b0-bcc9a62668d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:56.569942 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaace7e-d889-4039-a4b0-bcc9a62668d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:56.570217 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaace7e-d889-4039-a4b0-bcc9a62668d1 admin admin] PATH_INFO: `/v3/projects/a8d813d819034349b6fac43872a5d91e` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:56.571374 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaace7e-d889-4039-a4b0-bcc9a62668d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8d813d819034349b6fac43872a5d91e)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:56.573332 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaace7e-d889-4039-a4b0-bcc9a62668d1 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:56.619986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-deaace7e-d889-4039-a4b0-bcc9a62668d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8d813d819034349b6fac43872a5d91e', 'request_id': 'req-deaace7e-d889-4039-a4b0-bcc9a62668d1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:56.627548 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-deaace7e-d889-4039-a4b0-bcc9a62668d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8d813d819034349b6fac43872a5d91e', 'request_id': 'req-deaace7e-d889-4039-a4b0-bcc9a62668d1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:56.633021 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-deaace7e-d889-4039-a4b0-bcc9a62668d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8d813d819034349b6fac43872a5d91e', 'request_id': 'req-deaace7e-d889-4039-a4b0-bcc9a62668d1'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:56.644807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-deaace7e-d889-4039-a4b0-bcc9a62668d1 admin admin] The token cache is being invalidate because project a8d813d819034349b6fac43872a5d91e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:24:56.645093 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-deaace7e-d889-4039-a4b0-bcc9a62668d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:56.645755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-deaace7e-d889-4039-a4b0-bcc9a62668d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:56.647535 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1353/2716] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:24:56 2022] DELETE /identity/v3/projects/a8d813d819034349b6fac43872a5d91e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:56.651317 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fd877-11bf-41c1-8678-a83268f8c541 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:56.666798 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0fd877-11bf-41c1-8678-a83268f8c541 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:56.730056 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97ed3f-893d-4ecd-9770-e46cf2e25d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:56.731215 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97ed3f-893d-4ecd-9770-e46cf2e25d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:56.731738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97ed3f-893d-4ecd-9770-e46cf2e25d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:56.732214 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97ed3f-893d-4ecd-9770-e46cf2e25d6b admin admin] PATH_INFO: `/v3/users/6b6450f1dab24181aefccf2787dbb293` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:56.733644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97ed3f-893d-4ecd-9770-e46cf2e25d6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b6450f1dab24181aefccf2787dbb293)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:56.735579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97ed3f-893d-4ecd-9770-e46cf2e25d6b admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:56.755239 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:56.801118 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5f97ed3f-893d-4ecd-9770-e46cf2e25d6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b6450f1dab24181aefccf2787dbb293'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:56.814367 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5f97ed3f-893d-4ecd-9770-e46cf2e25d6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b6450f1dab24181aefccf2787dbb293'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:56.815151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5f97ed3f-893d-4ecd-9770-e46cf2e25d6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b6450f1dab24181aefccf2787dbb293'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:56.828805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5f97ed3f-893d-4ecd-9770-e46cf2e25d6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b6450f1dab24181aefccf2787dbb293'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:56.829684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-5f97ed3f-893d-4ecd-9770-e46cf2e25d6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b6450f1dab24181aefccf2787dbb293'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:56.842967 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1364/2717] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:24:56 2022] DELETE /identity/v3/users/6b6450f1dab24181aefccf2787dbb293 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:24:57.128507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaace7e-d889-4039-a4b0-bcc9a62668d1 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:57.129964 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-deaace7e-d889-4039-a4b0-bcc9a62668d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:57.145083 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaace7e-d889-4039-a4b0-bcc9a62668d1 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:57.159439 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:57.204930 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2ed19-f5e7-410f-a1a8-3985b13724fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:57.205506 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2ed19-f5e7-410f-a1a8-3985b13724fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:57.205776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2ed19-f5e7-410f-a1a8-3985b13724fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:57.206044 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2ed19-f5e7-410f-a1a8-3985b13724fa admin admin] PATH_INFO: `/v3/projects/c21f8b982e9d466ea00e47b8f3008aa7` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:57.208821 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2ed19-f5e7-410f-a1a8-3985b13724fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c21f8b982e9d466ea00e47b8f3008aa7)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:57.210720 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2ed19-f5e7-410f-a1a8-3985b13724fa admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:57.256952 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a4f2ed19-f5e7-410f-a1a8-3985b13724fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c21f8b982e9d466ea00e47b8f3008aa7', 'request_id': 'req-a4f2ed19-f5e7-410f-a1a8-3985b13724fa'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:57.264809 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a4f2ed19-f5e7-410f-a1a8-3985b13724fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c21f8b982e9d466ea00e47b8f3008aa7', 'request_id': 'req-a4f2ed19-f5e7-410f-a1a8-3985b13724fa'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:57.271166 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a4f2ed19-f5e7-410f-a1a8-3985b13724fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c21f8b982e9d466ea00e47b8f3008aa7', 'request_id': 'req-a4f2ed19-f5e7-410f-a1a8-3985b13724fa'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:57.282953 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a4f2ed19-f5e7-410f-a1a8-3985b13724fa admin admin] The token cache is being invalidate because project c21f8b982e9d466ea00e47b8f3008aa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:24:57.283243 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a4f2ed19-f5e7-410f-a1a8-3985b13724fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:57.283885 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-a4f2ed19-f5e7-410f-a1a8-3985b13724fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:24:57.285422 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1354/2718] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:24:57 2022] DELETE /identity/v3/projects/c21f8b982e9d466ea00e47b8f3008aa7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:57.290416 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43480f-f7d0-4452-a22b-82a54eb742bd None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:57.290883 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43480f-f7d0-4452-a22b-82a54eb742bd None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:57.290977 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43480f-f7d0-4452-a22b-82a54eb742bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:57.310780 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-fd43480f-f7d0-4452-a22b-82a54eb742bd None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:57.311984 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-fd43480f-f7d0-4452-a22b-82a54eb742bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:57.370154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1365/2719] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:57.378183 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43875cba-6551-47d9-8e12-d7f08e99309e None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:57.378466 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43875cba-6551-47d9-8e12-d7f08e99309e None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:57.378741 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43875cba-6551-47d9-8e12-d7f08e99309e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:57.398053 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-43875cba-6551-47d9-8e12-d7f08e99309e None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:57.399245 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-43875cba-6551-47d9-8e12-d7f08e99309e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:57.407030 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1355/2720] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:57.411777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43480f-f7d0-4452-a22b-82a54eb742bd None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:57.421519 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd43480f-f7d0-4452-a22b-82a54eb742bd None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:57.426737 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d4957-44dc-4b17-889f-99c1d72a0b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:57.427346 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d4957-44dc-4b17-889f-99c1d72a0b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:57.427687 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d4957-44dc-4b17-889f-99c1d72a0b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:57.427994 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d4957-44dc-4b17-889f-99c1d72a0b4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:57.429036 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d4957-44dc-4b17-889f-99c1d72a0b4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:24:57.429445 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d4957-44dc-4b17-889f-99c1d72a0b4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:57.431511 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d4957-44dc-4b17-889f-99c1d72a0b4b admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:57.444228 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1366/2721] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:24:57.448448 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43875cba-6551-47d9-8e12-d7f08e99309e None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:57.457753 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43875cba-6551-47d9-8e12-d7f08e99309e None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:57.462370 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fa849-0789-49a3-b88e-9ed51b876b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:57.462971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fa849-0789-49a3-b88e-9ed51b876b0e admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:57.463253 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fa849-0789-49a3-b88e-9ed51b876b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:57.463527 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fa849-0789-49a3-b88e-9ed51b876b0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:57.464181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fa849-0789-49a3-b88e-9ed51b876b0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:57.466107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fa849-0789-49a3-b88e-9ed51b876b0e admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:57.497248 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1356/2722] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:24:57 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:57.501438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d4957-44dc-4b17-889f-99c1d72a0b4b admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:57.511502 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d4957-44dc-4b17-889f-99c1d72a0b4b admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:57.569010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08633cd3-f37a-499d-8590-930509fe7437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:57.569624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08633cd3-f37a-499d-8590-930509fe7437 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:57.569902 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08633cd3-f37a-499d-8590-930509fe7437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:57.570177 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08633cd3-f37a-499d-8590-930509fe7437 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:57.570834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08633cd3-f37a-499d-8590-930509fe7437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:57.572752 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08633cd3-f37a-499d-8590-930509fe7437 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:57.593878 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1367/2723] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:24:57 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:57.598980 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fa849-0789-49a3-b88e-9ed51b876b0e admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:57.609390 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fa849-0789-49a3-b88e-9ed51b876b0e admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:57.614309 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e948f17-c8a7-4ee6-8d24-2b98394c95c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:57.614918 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e948f17-c8a7-4ee6-8d24-2b98394c95c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:57.615196 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e948f17-c8a7-4ee6-8d24-2b98394c95c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:57.615477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e948f17-c8a7-4ee6-8d24-2b98394c95c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:57.616167 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e948f17-c8a7-4ee6-8d24-2b98394c95c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:57.618681 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e948f17-c8a7-4ee6-8d24-2b98394c95c4 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:57.628841 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1357/2724] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:57.632861 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08633cd3-f37a-499d-8590-930509fe7437 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:57.643199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08633cd3-f37a-499d-8590-930509fe7437 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:57.648874 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718de772-8670-42ba-a2c9-1d808b604630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:57.649486 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718de772-8670-42ba-a2c9-1d808b604630 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:57.649769 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718de772-8670-42ba-a2c9-1d808b604630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:57.650044 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718de772-8670-42ba-a2c9-1d808b604630 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:57.650740 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718de772-8670-42ba-a2c9-1d808b604630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:57.652351 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718de772-8670-42ba-a2c9-1d808b604630 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:57.658986 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:57.665952 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1368/2725] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:57.670610 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e948f17-c8a7-4ee6-8d24-2b98394c95c4 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:57.680703 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e948f17-c8a7-4ee6-8d24-2b98394c95c4 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:57.685236 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ed526-6e26-49a3-9a6c-777436c4b71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:57.685889 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ed526-6e26-49a3-9a6c-777436c4b71e admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:57.686200 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ed526-6e26-49a3-9a6c-777436c4b71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:57.686473 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ed526-6e26-49a3-9a6c-777436c4b71e admin admin] PATH_INFO: `/v3/projects/6667189d0e7b4f1b992e2fedfefd97c1/users/38b3cb1e7735492ead89c034bd17646c/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:57.701233 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ed526-6e26-49a3-9a6c-777436c4b71e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6667189d0e7b4f1b992e2fedfefd97c1, user_id=38b3cb1e7735492ead89c034bd17646c, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:57.703899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ed526-6e26-49a3-9a6c-777436c4b71e admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:57.729102 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1358/2726] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:57 2022] PUT /identity/v3/projects/6667189d0e7b4f1b992e2fedfefd97c1/users/38b3cb1e7735492ead89c034bd17646c/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:57.739336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e29de-8053-4f86-8bce-e451887b4ad0 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:57.739620 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e29de-8053-4f86-8bce-e451887b4ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:57.739896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0e29de-8053-4f86-8bce-e451887b4ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:57.777198 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-0e0e29de-8053-4f86-8bce-e451887b4ad0 None None] MFA Rules not processed for user `38b3cb1e7735492ead89c034bd17646c`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:57.778438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-0e0e29de-8053-4f86-8bce-e451887b4ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:57.860542 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1369/2727] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:57.864970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ed526-6e26-49a3-9a6c-777436c4b71e admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:57.902817 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:57.923805 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c7233-20da-44f4-9631-29da67f54a46 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:57.924326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c7233-20da-44f4-9631-29da67f54a46 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:57.924712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c7233-20da-44f4-9631-29da67f54a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:57.944743 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-4f6c7233-20da-44f4-9631-29da67f54a46 None None] MFA Rules not processed for user `b18a6ae879c14eabae7a7a6db120831a`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:57.945939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4f6c7233-20da-44f4-9631-29da67f54a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:57.979376 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ed526-6e26-49a3-9a6c-777436c4b71e admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:57.983954 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c592ddf-c6f9-470d-83db-dc3ed8e9a598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:57.984581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c592ddf-c6f9-470d-83db-dc3ed8e9a598 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:57.984905 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c592ddf-c6f9-470d-83db-dc3ed8e9a598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:57.985178 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c592ddf-c6f9-470d-83db-dc3ed8e9a598 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:57.985901 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c592ddf-c6f9-470d-83db-dc3ed8e9a598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:57.987821 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c592ddf-c6f9-470d-83db-dc3ed8e9a598 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:58.004339 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1370/2728] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:58.023769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1359/2729] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:24:57 2022] POST /identity/v3/projects => generated 385 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:58.027751 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c7233-20da-44f4-9631-29da67f54a46 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:58.041582 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6c7233-20da-44f4-9631-29da67f54a46 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:58.115709 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b19edd-6a5f-4d3f-b330-275a7fd0a03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:58.116326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19edd-6a5f-4d3f-b330-275a7fd0a03b admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:58.116608 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19edd-6a5f-4d3f-b330-275a7fd0a03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:58.116909 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b19edd-6a5f-4d3f-b330-275a7fd0a03b admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:58.117580 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b19edd-6a5f-4d3f-b330-275a7fd0a03b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:58.119510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b19edd-6a5f-4d3f-b330-275a7fd0a03b admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:58.138969 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:58.146592 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1371/2730] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:24:58 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:58.149870 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c592ddf-c6f9-470d-83db-dc3ed8e9a598 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:58.160413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c592ddf-c6f9-470d-83db-dc3ed8e9a598 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:58.165034 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9440fc-2d0c-4b21-934c-a101ee8e356b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:58.165680 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9440fc-2d0c-4b21-934c-a101ee8e356b admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:58.165957 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9440fc-2d0c-4b21-934c-a101ee8e356b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:58.166237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9440fc-2d0c-4b21-934c-a101ee8e356b admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:58.166936 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9440fc-2d0c-4b21-934c-a101ee8e356b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:58.168526 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9440fc-2d0c-4b21-934c-a101ee8e356b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:58.178776 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1360/2731] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:24:58.182885 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b19edd-6a5f-4d3f-b330-275a7fd0a03b admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:58.193947 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b19edd-6a5f-4d3f-b330-275a7fd0a03b admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:58.198220 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ac409-13d0-4285-b05d-d6ebfa8d9932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:58.198828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ac409-13d0-4285-b05d-d6ebfa8d9932 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:58.199112 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ac409-13d0-4285-b05d-d6ebfa8d9932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:58.199393 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ac409-13d0-4285-b05d-d6ebfa8d9932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:58.200088 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ac409-13d0-4285-b05d-d6ebfa8d9932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:58.201704 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ac409-13d0-4285-b05d-d6ebfa8d9932 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:58.215885 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1372/2732] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:58.217234 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9440fc-2d0c-4b21-934c-a101ee8e356b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:58.227649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9440fc-2d0c-4b21-934c-a101ee8e356b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:58.232299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d667a-afa7-4a9c-834b-7b9024f89cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:58.232921 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d667a-afa7-4a9c-834b-7b9024f89cca admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:58.233197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d667a-afa7-4a9c-834b-7b9024f89cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:58.233471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d667a-afa7-4a9c-834b-7b9024f89cca admin admin] PATH_INFO: `/v3/projects/5f08b61b7e9148ebb8d3fb78ca2b8a9d/users/5ae87c871f8a491fb661be7b79089b07/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:58.257002 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d667a-afa7-4a9c-834b-7b9024f89cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f08b61b7e9148ebb8d3fb78ca2b8a9d, user_id=5ae87c871f8a491fb661be7b79089b07, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:58.259658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d667a-afa7-4a9c-834b-7b9024f89cca admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:58.274602 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1361/2733] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:58 2022] PUT /identity/v3/projects/5f08b61b7e9148ebb8d3fb78ca2b8a9d/users/5ae87c871f8a491fb661be7b79089b07/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:58.286123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0db7fd-3b06-4de9-9af1-0442fdeea544 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:58.286408 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0db7fd-3b06-4de9-9af1-0442fdeea544 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:58.286707 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0db7fd-3b06-4de9-9af1-0442fdeea544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:58.316681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-ef0db7fd-3b06-4de9-9af1-0442fdeea544 None None] MFA Rules not processed for user `5ae87c871f8a491fb661be7b79089b07`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:58.318050 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-ef0db7fd-3b06-4de9-9af1-0442fdeea544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:58.384451 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1373/2734] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:24:58.389352 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d667a-afa7-4a9c-834b-7b9024f89cca admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:58.453088 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d667a-afa7-4a9c-834b-7b9024f89cca admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:58.458206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94f975-83e0-4702-8ee1-275dc8fcfa69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:58.458821 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94f975-83e0-4702-8ee1-275dc8fcfa69 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:58.459099 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94f975-83e0-4702-8ee1-275dc8fcfa69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:58.459368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94f975-83e0-4702-8ee1-275dc8fcfa69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:58.460030 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94f975-83e0-4702-8ee1-275dc8fcfa69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:58.462014 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94f975-83e0-4702-8ee1-275dc8fcfa69 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:58.490203 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1362/2735] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:24:58 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:24:58.494436 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0db7fd-3b06-4de9-9af1-0442fdeea544 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:58.505862 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0db7fd-3b06-4de9-9af1-0442fdeea544 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:58.577081 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd21937-90b7-4470-9826-694b62058b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:58.577979 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd21937-90b7-4470-9826-694b62058b31 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:58.578330 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd21937-90b7-4470-9826-694b62058b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:58.578677 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd21937-90b7-4470-9826-694b62058b31 admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:58.579452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd21937-90b7-4470-9826-694b62058b31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:58.581513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd21937-90b7-4470-9826-694b62058b31 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:58.607499 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1374/2736] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:24:58 2022] POST /identity/v3/users => generated 299 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:24:58.612348 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94f975-83e0-4702-8ee1-275dc8fcfa69 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:58.624877 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94f975-83e0-4702-8ee1-275dc8fcfa69 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:58.631828 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a5a3b-cc45-4112-8c92-c564a1817a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:58.632536 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a5a3b-cc45-4112-8c92-c564a1817a99 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:58.632945 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a5a3b-cc45-4112-8c92-c564a1817a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:58.633312 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a5a3b-cc45-4112-8c92-c564a1817a99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:58.634585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a5a3b-cc45-4112-8c92-c564a1817a99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:58.648881 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a5a3b-cc45-4112-8c92-c564a1817a99 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:58.668978 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1363/2737] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:58.671187 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd21937-90b7-4470-9826-694b62058b31 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:58.682719 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd21937-90b7-4470-9826-694b62058b31 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:58.687405 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c6253-6ab2-4319-9e22-0407077c75ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:58.688069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c6253-6ab2-4319-9e22-0407077c75ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:58.688391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c6253-6ab2-4319-9e22-0407077c75ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:58.688757 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c6253-6ab2-4319-9e22-0407077c75ec admin admin] PATH_INFO: `/v3/projects/e3b01b0bcf7040edaca85ecf374edad3/users/e10fc464ea384f49986c25f0bb6f7284/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:58.704516 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c6253-6ab2-4319-9e22-0407077c75ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3b01b0bcf7040edaca85ecf374edad3, user_id=e10fc464ea384f49986c25f0bb6f7284, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:58.707623 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c6253-6ab2-4319-9e22-0407077c75ec admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:58.725634 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1375/2738] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:58 2022] PUT /identity/v3/projects/e3b01b0bcf7040edaca85ecf374edad3/users/e10fc464ea384f49986c25f0bb6f7284/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:24:58.728909 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a5a3b-cc45-4112-8c92-c564a1817a99 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:58.740176 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a5a3b-cc45-4112-8c92-c564a1817a99 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:58.752663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:58.803928 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7cc3f7-877a-4e08-8f22-affbd5ca13a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:58.804555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7cc3f7-877a-4e08-8f22-affbd5ca13a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:58.804867 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7cc3f7-877a-4e08-8f22-affbd5ca13a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:58.805144 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7cc3f7-877a-4e08-8f22-affbd5ca13a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:58.805839 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7cc3f7-877a-4e08-8f22-affbd5ca13a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:58.807437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7cc3f7-877a-4e08-8f22-affbd5ca13a0 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:58.818190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1364/2739] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:24:58.822873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c6253-6ab2-4319-9e22-0407077c75ec admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:58.835308 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c6253-6ab2-4319-9e22-0407077c75ec admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:58.840733 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade3d61-6467-4324-8e4b-7a01bf83870d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:58.841404 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade3d61-6467-4324-8e4b-7a01bf83870d admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:58.841690 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade3d61-6467-4324-8e4b-7a01bf83870d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:58.841967 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade3d61-6467-4324-8e4b-7a01bf83870d admin admin] PATH_INFO: `/v3/projects/e3b01b0bcf7040edaca85ecf374edad3/users/e10fc464ea384f49986c25f0bb6f7284/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:58.844068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade3d61-6467-4324-8e4b-7a01bf83870d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3b01b0bcf7040edaca85ecf374edad3, user_id=e10fc464ea384f49986c25f0bb6f7284, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:58.847518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fade3d61-6467-4324-8e4b-7a01bf83870d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:58.859477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1376/2740] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:24:58 2022] PUT /identity/v3/projects/e3b01b0bcf7040edaca85ecf374edad3/users/e10fc464ea384f49986c25f0bb6f7284/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:24:58.870413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea8e54-e0b3-40c6-a72f-1f1c14662ccf None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:58.870815 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea8e54-e0b3-40c6-a72f-1f1c14662ccf None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:58.871163 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea8e54-e0b3-40c6-a72f-1f1c14662ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:58.905467 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-d9ea8e54-e0b3-40c6-a72f-1f1c14662ccf None None] MFA Rules not processed for user `e10fc464ea384f49986c25f0bb6f7284`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:24:58.907113 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d9ea8e54-e0b3-40c6-a72f-1f1c14662ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:58.990552 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1365/2741] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:24:58 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:24:59.001766 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade3d61-6467-4324-8e4b-7a01bf83870d admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:59.003547 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-fade3d61-6467-4324-8e4b-7a01bf83870d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:59.038076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:24:59.083481 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade3d61-6467-4324-8e4b-7a01bf83870d admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:59.087231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9ca59-633f-400a-8d86-8e24a1dd171f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:59.087870 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9ca59-633f-400a-8d86-8e24a1dd171f service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:59.088153 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9ca59-633f-400a-8d86-8e24a1dd171f service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:59.088428 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9ca59-633f-400a-8d86-8e24a1dd171f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:59.099131 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9ca59-633f-400a-8d86-8e24a1dd171f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:59.100564 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9ca59-633f-400a-8d86-8e24a1dd171f service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:59.110991 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1377/2742] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:24:59.645135 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea8e54-e0b3-40c6-a72f-1f1c14662ccf None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:24:59.646567 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d9ea8e54-e0b3-40c6-a72f-1f1c14662ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:24:59.658868 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea8e54-e0b3-40c6-a72f-1f1c14662ccf None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:24:59.662677 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fe101-aa8e-4aa7-81d6-aae155c0c0a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:24:59.663293 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fe101-aa8e-4aa7-81d6-aae155c0c0a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:24:59.663583 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fe101-aa8e-4aa7-81d6-aae155c0c0a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:24:59.663833 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18fe101-aa8e-4aa7-81d6-aae155c0c0a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:24:59.723251 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18fe101-aa8e-4aa7-81d6-aae155c0c0a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:24:59.724736 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18fe101-aa8e-4aa7-81d6-aae155c0c0a9 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:24:59.734480 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1366/2743] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:24:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:25:01.694402 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9ca59-633f-400a-8d86-8e24a1dd171f service cinder] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:01.696946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-18e9ca59-633f-400a-8d86-8e24a1dd171f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:01.764744 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9ca59-633f-400a-8d86-8e24a1dd171f service cinder] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:01.768778 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eefc86-abd8-4269-b741-a1578d92f96d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:01.769418 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eefc86-abd8-4269-b741-a1578d92f96d service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:01.769684 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eefc86-abd8-4269-b741-a1578d92f96d service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:01.769963 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eefc86-abd8-4269-b741-a1578d92f96d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:01.771698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e3eefc86-abd8-4269-b741-a1578d92f96d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:01.834797 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eefc86-abd8-4269-b741-a1578d92f96d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:01.836231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eefc86-abd8-4269-b741-a1578d92f96d service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:01.846783 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1378/2744] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:25:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:25:03.850533 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fe101-aa8e-4aa7-81d6-aae155c0c0a9 service cinder] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:03.860825 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18fe101-aa8e-4aa7-81d6-aae155c0c0a9 service cinder] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:03.865830 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1242f7d-d86e-4fc8-8365-f482c0bbfcb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:03.866648 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1242f7d-d86e-4fc8-8365-f482c0bbfcb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:03.867035 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1242f7d-d86e-4fc8-8365-f482c0bbfcb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:03.867436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1242f7d-d86e-4fc8-8365-f482c0bbfcb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:03.908569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=100 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:03.945174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1242f7d-d86e-4fc8-8365-f482c0bbfcb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:03.947074 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1242f7d-d86e-4fc8-8365-f482c0bbfcb4 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:03.960220 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1367/2745] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:25:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:25:06.684291 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eefc86-abd8-4269-b741-a1578d92f96d service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:06.760131 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eefc86-abd8-4269-b741-a1578d92f96d service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:06.766535 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8d8ad-4f35-4a77-ae29-7d009f963585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:06.767528 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8d8ad-4f35-4a77-ae29-7d009f963585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:06.767858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8d8ad-4f35-4a77-ae29-7d009f963585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:06.768135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a8d8ad-4f35-4a77-ae29-7d009f963585 admin admin] PATH_INFO: `/v3/users/38b3cb1e7735492ead89c034bd17646c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:06.769499 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a8d8ad-4f35-4a77-ae29-7d009f963585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38b3cb1e7735492ead89c034bd17646c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:06.771417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a8d8ad-4f35-4a77-ae29-7d009f963585 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:06.781080 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:06.830477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-e3a8d8ad-4f35-4a77-ae29-7d009f963585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38b3cb1e7735492ead89c034bd17646c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:06.839436 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-e3a8d8ad-4f35-4a77-ae29-7d009f963585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38b3cb1e7735492ead89c034bd17646c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:06.840170 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-e3a8d8ad-4f35-4a77-ae29-7d009f963585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38b3cb1e7735492ead89c034bd17646c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:06.855073 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-e3a8d8ad-4f35-4a77-ae29-7d009f963585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38b3cb1e7735492ead89c034bd17646c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:06.856047 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-e3a8d8ad-4f35-4a77-ae29-7d009f963585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38b3cb1e7735492ead89c034bd17646c'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:06.870738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1379/2746] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:25:06 2022] DELETE /identity/v3/users/38b3cb1e7735492ead89c034bd17646c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:06.875263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386859db-51b1-4df4-93e8-91de8702fa89 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:06.876368 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386859db-51b1-4df4-93e8-91de8702fa89 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:06.876486 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386859db-51b1-4df4-93e8-91de8702fa89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:06.903951 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-386859db-51b1-4df4-93e8-91de8702fa89 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:06.905202 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-386859db-51b1-4df4-93e8-91de8702fa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:06.966381 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1368/2747] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:06.977227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8d8ad-4f35-4a77-ae29-7d009f963585 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:06.978723 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e3a8d8ad-4f35-4a77-ae29-7d009f963585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:07.053074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a8d8ad-4f35-4a77-ae29-7d009f963585 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:07.058983 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0dbd5-04b4-473f-9fd6-5d5c7f83bc9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:07.059908 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0dbd5-04b4-473f-9fd6-5d5c7f83bc9d service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:07.060328 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0dbd5-04b4-473f-9fd6-5d5c7f83bc9d service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:07.060781 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0dbd5-04b4-473f-9fd6-5d5c7f83bc9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:07.073979 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb0dbd5-04b4-473f-9fd6-5d5c7f83bc9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:07.075558 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb0dbd5-04b4-473f-9fd6-5d5c7f83bc9d service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:07.087312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1380/2748] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:25:07.378476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386859db-51b1-4df4-93e8-91de8702fa89 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:07.380515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-386859db-51b1-4df4-93e8-91de8702fa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:07.403239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386859db-51b1-4df4-93e8-91de8702fa89 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:07.408555 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffddd102-0026-47b6-9337-5b6a0a47165e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:07.409267 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddd102-0026-47b6-9337-5b6a0a47165e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:07.409544 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddd102-0026-47b6-9337-5b6a0a47165e admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:07.409814 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddd102-0026-47b6-9337-5b6a0a47165e admin admin] PATH_INFO: `/v3/projects/6667189d0e7b4f1b992e2fedfefd97c1` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:07.411019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffddd102-0026-47b6-9337-5b6a0a47165e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6667189d0e7b4f1b992e2fedfefd97c1)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:07.412883 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffddd102-0026-47b6-9337-5b6a0a47165e admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:07.460381 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ffddd102-0026-47b6-9337-5b6a0a47165e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6667189d0e7b4f1b992e2fedfefd97c1', 'request_id': 'req-ffddd102-0026-47b6-9337-5b6a0a47165e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:07.468733 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ffddd102-0026-47b6-9337-5b6a0a47165e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6667189d0e7b4f1b992e2fedfefd97c1', 'request_id': 'req-ffddd102-0026-47b6-9337-5b6a0a47165e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:07.475626 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ffddd102-0026-47b6-9337-5b6a0a47165e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6667189d0e7b4f1b992e2fedfefd97c1', 'request_id': 'req-ffddd102-0026-47b6-9337-5b6a0a47165e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:07.488038 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ffddd102-0026-47b6-9337-5b6a0a47165e admin admin] The token cache is being invalidate because project 6667189d0e7b4f1b992e2fedfefd97c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:25:07.488338 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ffddd102-0026-47b6-9337-5b6a0a47165e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:07.489099 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ffddd102-0026-47b6-9337-5b6a0a47165e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:07.491226 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1369/2749] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:25:07 2022] DELETE /identity/v3/projects/6667189d0e7b4f1b992e2fedfefd97c1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:25:07.495279 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0dbd5-04b4-473f-9fd6-5d5c7f83bc9d service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:07.533055 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0dbd5-04b4-473f-9fd6-5d5c7f83bc9d service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:07.609901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510c36f-1045-446a-ba19-d648a0af30aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:07.610132 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510c36f-1045-446a-ba19-d648a0af30aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:07.610458 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510c36f-1045-446a-ba19-d648a0af30aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:07.610745 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4510c36f-1045-446a-ba19-d648a0af30aa admin admin] PATH_INFO: `/v3/users/5ae87c871f8a491fb661be7b79089b07` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:07.611988 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510c36f-1045-446a-ba19-d648a0af30aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ae87c871f8a491fb661be7b79089b07)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:07.614069 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4510c36f-1045-446a-ba19-d648a0af30aa admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:07.665831 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:07.683960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4510c36f-1045-446a-ba19-d648a0af30aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ae87c871f8a491fb661be7b79089b07'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:07.701000 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4510c36f-1045-446a-ba19-d648a0af30aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ae87c871f8a491fb661be7b79089b07'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:07.702681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4510c36f-1045-446a-ba19-d648a0af30aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ae87c871f8a491fb661be7b79089b07'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:07.720314 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4510c36f-1045-446a-ba19-d648a0af30aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ae87c871f8a491fb661be7b79089b07'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:07.721478 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4510c36f-1045-446a-ba19-d648a0af30aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ae87c871f8a491fb661be7b79089b07'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:07.748809 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1381/2750] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:25:07 2022] DELETE /identity/v3/users/5ae87c871f8a491fb661be7b79089b07 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:08.060442 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffddd102-0026-47b6-9337-5b6a0a47165e admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:08.062150 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ffddd102-0026-47b6-9337-5b6a0a47165e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:08.078765 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffddd102-0026-47b6-9337-5b6a0a47165e admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:08.132875 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:08.144574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d481001-abb7-48bc-b66b-d186eb4f662e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:08.145481 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d481001-abb7-48bc-b66b-d186eb4f662e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:08.145950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d481001-abb7-48bc-b66b-d186eb4f662e admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:08.146348 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d481001-abb7-48bc-b66b-d186eb4f662e admin admin] PATH_INFO: `/v3/projects/5f08b61b7e9148ebb8d3fb78ca2b8a9d` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:08.147696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d481001-abb7-48bc-b66b-d186eb4f662e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f08b61b7e9148ebb8d3fb78ca2b8a9d)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:08.149638 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d481001-abb7-48bc-b66b-d186eb4f662e admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:08.195831 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-3d481001-abb7-48bc-b66b-d186eb4f662e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f08b61b7e9148ebb8d3fb78ca2b8a9d', 'request_id': 'req-3d481001-abb7-48bc-b66b-d186eb4f662e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:08.202939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-3d481001-abb7-48bc-b66b-d186eb4f662e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f08b61b7e9148ebb8d3fb78ca2b8a9d', 'request_id': 'req-3d481001-abb7-48bc-b66b-d186eb4f662e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:08.209254 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-3d481001-abb7-48bc-b66b-d186eb4f662e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f08b61b7e9148ebb8d3fb78ca2b8a9d', 'request_id': 'req-3d481001-abb7-48bc-b66b-d186eb4f662e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:08.220988 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-3d481001-abb7-48bc-b66b-d186eb4f662e admin admin] The token cache is being invalidate because project 5f08b61b7e9148ebb8d3fb78ca2b8a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:25:08.221278 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-3d481001-abb7-48bc-b66b-d186eb4f662e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:08.221929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-3d481001-abb7-48bc-b66b-d186eb4f662e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:08.223426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1370/2751] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:25:08 2022] DELETE /identity/v3/projects/5f08b61b7e9148ebb8d3fb78ca2b8a9d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:08.227381 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510c36f-1045-446a-ba19-d648a0af30aa admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:08.228816 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4510c36f-1045-446a-ba19-d648a0af30aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:08.242162 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4510c36f-1045-446a-ba19-d648a0af30aa admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:08.301985 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08c070-518f-42f4-837d-9686fb5171f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:08.302691 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08c070-518f-42f4-837d-9686fb5171f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:08.302980 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08c070-518f-42f4-837d-9686fb5171f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:08.303257 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f08c070-518f-42f4-837d-9686fb5171f6 admin admin] PATH_INFO: `/v3/users/e10fc464ea384f49986c25f0bb6f7284` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:08.304435 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08c070-518f-42f4-837d-9686fb5171f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e10fc464ea384f49986c25f0bb6f7284)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:08.306391 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f08c070-518f-42f4-837d-9686fb5171f6 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:08.370558 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2f08c070-518f-42f4-837d-9686fb5171f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e10fc464ea384f49986c25f0bb6f7284'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:08.381306 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2f08c070-518f-42f4-837d-9686fb5171f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e10fc464ea384f49986c25f0bb6f7284'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:08.382007 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2f08c070-518f-42f4-837d-9686fb5171f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e10fc464ea384f49986c25f0bb6f7284'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:08.394602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2f08c070-518f-42f4-837d-9686fb5171f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e10fc464ea384f49986c25f0bb6f7284'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:08.395459 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-2f08c070-518f-42f4-837d-9686fb5171f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e10fc464ea384f49986c25f0bb6f7284'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:08.409668 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1382/2752] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:25:08 2022] DELETE /identity/v3/users/e10fc464ea384f49986c25f0bb6f7284 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:08.592810 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d481001-abb7-48bc-b66b-d186eb4f662e admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:08.595469 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-3d481001-abb7-48bc-b66b-d186eb4f662e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:08.707013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d481001-abb7-48bc-b66b-d186eb4f662e admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:08.767890 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08c070-518f-42f4-837d-9686fb5171f6 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:08.769606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-2f08c070-518f-42f4-837d-9686fb5171f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:08.809474 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d14e1f-513d-4641-90bd-ada9357466f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796772cf491047c8b0a609dae9402307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:08.810372 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d14e1f-513d-4641-90bd-ada9357466f5 service placement] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:08.810758 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d14e1f-513d-4641-90bd-ada9357466f5 service placement] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:08.811184 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d14e1f-513d-4641-90bd-ada9357466f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:08.813616 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-37d14e1f-513d-4641-90bd-ada9357466f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:08.826557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:08.830010 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:08.833604 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d14e1f-513d-4641-90bd-ada9357466f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:08.842880 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d14e1f-513d-4641-90bd-ada9357466f5 service placement] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:08.865296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1371/2753] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:25:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:25:08.882813 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f08c070-518f-42f4-837d-9686fb5171f6 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:08.890761 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e446fb-e51e-4cb9-af0e-9aad64c7f151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:08.892371 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e446fb-e51e-4cb9-af0e-9aad64c7f151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:08.892893 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e446fb-e51e-4cb9-af0e-9aad64c7f151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:08.893349 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e446fb-e51e-4cb9-af0e-9aad64c7f151 admin admin] PATH_INFO: `/v3/projects/e3b01b0bcf7040edaca85ecf374edad3` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:08.895306 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e446fb-e51e-4cb9-af0e-9aad64c7f151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3b01b0bcf7040edaca85ecf374edad3)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:08.898810 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e446fb-e51e-4cb9-af0e-9aad64c7f151 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:08.974360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-e8e446fb-e51e-4cb9-af0e-9aad64c7f151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3b01b0bcf7040edaca85ecf374edad3', 'request_id': 'req-e8e446fb-e51e-4cb9-af0e-9aad64c7f151'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:08.983599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-e8e446fb-e51e-4cb9-af0e-9aad64c7f151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3b01b0bcf7040edaca85ecf374edad3', 'request_id': 'req-e8e446fb-e51e-4cb9-af0e-9aad64c7f151'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:08.991231 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-e8e446fb-e51e-4cb9-af0e-9aad64c7f151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3b01b0bcf7040edaca85ecf374edad3', 'request_id': 'req-e8e446fb-e51e-4cb9-af0e-9aad64c7f151'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:09.005055 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-e8e446fb-e51e-4cb9-af0e-9aad64c7f151 admin admin] The token cache is being invalidate because project e3b01b0bcf7040edaca85ecf374edad3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:25:09.005344 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-e8e446fb-e51e-4cb9-af0e-9aad64c7f151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:09.006014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-e8e446fb-e51e-4cb9-af0e-9aad64c7f151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:09.007560 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1383/2754] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:25:08 2022] DELETE /identity/v3/projects/e3b01b0bcf7040edaca85ecf374edad3 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:25:09.013874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20acee-84f3-4866-965e-dd6372f987c2 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:09.014095 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20acee-84f3-4866-965e-dd6372f987c2 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:09.014369 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20acee-84f3-4866-965e-dd6372f987c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:09.096909 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-dd20acee-84f3-4866-965e-dd6372f987c2 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:09.098085 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-dd20acee-84f3-4866-965e-dd6372f987c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:09.161149 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1372/2755] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:09.169885 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e9c08-2735-4d9c-b8c5-66a660a37549 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:09.170190 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e9c08-2735-4d9c-b8c5-66a660a37549 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:09.170491 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e9c08-2735-4d9c-b8c5-66a660a37549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:09.190724 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-9e6e9c08-2735-4d9c-b8c5-66a660a37549 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:09.192042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9e6e9c08-2735-4d9c-b8c5-66a660a37549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:09.200247 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1384/2756] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:09.206649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20acee-84f3-4866-965e-dd6372f987c2 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:09.218939 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20acee-84f3-4866-965e-dd6372f987c2 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:09.223904 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a474e78-c3b6-4c34-9fcf-6b5538ba112e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:09.224525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a474e78-c3b6-4c34-9fcf-6b5538ba112e admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:09.224847 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a474e78-c3b6-4c34-9fcf-6b5538ba112e admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:09.225123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a474e78-c3b6-4c34-9fcf-6b5538ba112e admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:09.225819 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a474e78-c3b6-4c34-9fcf-6b5538ba112e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:25:09.226122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a474e78-c3b6-4c34-9fcf-6b5538ba112e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:09.227718 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a474e78-c3b6-4c34-9fcf-6b5538ba112e admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:09.239400 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1373/2757] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:25:09.243848 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e9c08-2735-4d9c-b8c5-66a660a37549 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:09.253742 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e9c08-2735-4d9c-b8c5-66a660a37549 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:09.258465 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac130ce8-2a64-41ce-9e3c-3512d60dff59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:09.259078 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac130ce8-2a64-41ce-9e3c-3512d60dff59 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:09.259431 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac130ce8-2a64-41ce-9e3c-3512d60dff59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:09.259694 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac130ce8-2a64-41ce-9e3c-3512d60dff59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:09.260334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac130ce8-2a64-41ce-9e3c-3512d60dff59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:09.262265 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac130ce8-2a64-41ce-9e3c-3512d60dff59 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:09.290844 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1385/2758] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:25:09 2022] POST /identity/v3/projects => generated 407 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:25:09.295084 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a474e78-c3b6-4c34-9fcf-6b5538ba112e admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:09.305790 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a474e78-c3b6-4c34-9fcf-6b5538ba112e admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:09.365500 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd72b21-fa02-4af7-90a2-a06da9f2b30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:09.366122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd72b21-fa02-4af7-90a2-a06da9f2b30b admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:09.366402 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd72b21-fa02-4af7-90a2-a06da9f2b30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:09.366676 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd72b21-fa02-4af7-90a2-a06da9f2b30b admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:09.367326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd72b21-fa02-4af7-90a2-a06da9f2b30b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:09.369271 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd72b21-fa02-4af7-90a2-a06da9f2b30b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:09.393458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1374/2759] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:25:09 2022] POST /identity/v3/users => generated 304 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:25:09.399501 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac130ce8-2a64-41ce-9e3c-3512d60dff59 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:09.409945 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac130ce8-2a64-41ce-9e3c-3512d60dff59 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:09.414621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529e969-3542-4695-8ec4-eef0cb0677f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:09.415225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529e969-3542-4695-8ec4-eef0cb0677f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:09.415536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529e969-3542-4695-8ec4-eef0cb0677f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:09.415837 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529e969-3542-4695-8ec4-eef0cb0677f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:09.416553 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529e969-3542-4695-8ec4-eef0cb0677f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:09.418189 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529e969-3542-4695-8ec4-eef0cb0677f8 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:09.428388 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1386/2760] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:25:09.433194 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd72b21-fa02-4af7-90a2-a06da9f2b30b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:09.444877 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd72b21-fa02-4af7-90a2-a06da9f2b30b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:09.449564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a264415d-fb83-4226-aae6-faaa4a7a808d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:09.450173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264415d-fb83-4226-aae6-faaa4a7a808d admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:09.450455 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264415d-fb83-4226-aae6-faaa4a7a808d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:09.450733 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a264415d-fb83-4226-aae6-faaa4a7a808d admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:09.451427 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264415d-fb83-4226-aae6-faaa4a7a808d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:09.453081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a264415d-fb83-4226-aae6-faaa4a7a808d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:09.465710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1375/2761] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:25:09.469945 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529e969-3542-4695-8ec4-eef0cb0677f8 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:09.480237 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529e969-3542-4695-8ec4-eef0cb0677f8 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:09.485602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce347a-2882-4096-b848-abcf26bf541e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:09.486255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce347a-2882-4096-b848-abcf26bf541e admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:09.486537 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce347a-2882-4096-b848-abcf26bf541e admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:09.486834 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce347a-2882-4096-b848-abcf26bf541e admin admin] PATH_INFO: `/v3/projects/c21917f9c96d49eda52b24d71a6c7d82/users/4bdfa1b58d9843ccb41bf84e246dc861/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:09.502574 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce347a-2882-4096-b848-abcf26bf541e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c21917f9c96d49eda52b24d71a6c7d82, user_id=4bdfa1b58d9843ccb41bf84e246dc861, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:09.505353 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce347a-2882-4096-b848-abcf26bf541e admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:09.518623 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1387/2762] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:25:09 2022] PUT /identity/v3/projects/c21917f9c96d49eda52b24d71a6c7d82/users/4bdfa1b58d9843ccb41bf84e246dc861/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:09.529876 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4072651-9cd5-4398-bd4f-a97162168040 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:09.530101 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4072651-9cd5-4398-bd4f-a97162168040 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:09.530379 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4072651-9cd5-4398-bd4f-a97162168040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:09.559023 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-d4072651-9cd5-4398-bd4f-a97162168040 None None] MFA Rules not processed for user `4bdfa1b58d9843ccb41bf84e246dc861`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:09.560237 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-d4072651-9cd5-4398-bd4f-a97162168040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:09.584222 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:09.625406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1376/2763] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:09 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:09.630492 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce347a-2882-4096-b848-abcf26bf541e admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:09.682855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:09.695292 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce347a-2882-4096-b848-abcf26bf541e admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:09.700079 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aad03d-5759-41a6-b7c8-332c311f69c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:09.700717 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aad03d-5759-41a6-b7c8-332c311f69c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:09.701001 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aad03d-5759-41a6-b7c8-332c311f69c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:09.701277 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aad03d-5759-41a6-b7c8-332c311f69c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:09.701938 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aad03d-5759-41a6-b7c8-332c311f69c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:09.703895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aad03d-5759-41a6-b7c8-332c311f69c9 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:09.731983 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1388/2764] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:25:09 2022] POST /identity/v3/projects => generated 407 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:25:09.736786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4072651-9cd5-4398-bd4f-a97162168040 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:09.748785 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4072651-9cd5-4398-bd4f-a97162168040 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:09.804836 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee27302-1a15-45fa-95df-2b5d6191db3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:09.805419 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee27302-1a15-45fa-95df-2b5d6191db3d admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:09.805697 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee27302-1a15-45fa-95df-2b5d6191db3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:09.805968 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee27302-1a15-45fa-95df-2b5d6191db3d admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:09.806613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee27302-1a15-45fa-95df-2b5d6191db3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:09.808495 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee27302-1a15-45fa-95df-2b5d6191db3d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:09.839262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1377/2765] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:25:09 2022] POST /identity/v3/users => generated 310 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:25:09.843437 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aad03d-5759-41a6-b7c8-332c311f69c9 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:09.854093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aad03d-5759-41a6-b7c8-332c311f69c9 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:09.858553 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c8aaa-1ffc-4b37-959f-46c550b9f5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:09.859155 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c8aaa-1ffc-4b37-959f-46c550b9f5ce admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:09.859429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c8aaa-1ffc-4b37-959f-46c550b9f5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:09.859701 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c8aaa-1ffc-4b37-959f-46c550b9f5ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:09.860937 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c8aaa-1ffc-4b37-959f-46c550b9f5ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:09.862505 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c8aaa-1ffc-4b37-959f-46c550b9f5ce admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:09.873394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1389/2766] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:25:09.879947 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee27302-1a15-45fa-95df-2b5d6191db3d admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:09.889795 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee27302-1a15-45fa-95df-2b5d6191db3d admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:09.894306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4167bd-c77b-4415-a6b1-1e0a8f66db4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:09.894944 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4167bd-c77b-4415-a6b1-1e0a8f66db4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:09.895219 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4167bd-c77b-4415-a6b1-1e0a8f66db4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:09.895505 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4167bd-c77b-4415-a6b1-1e0a8f66db4f admin admin] PATH_INFO: `/v3/projects/9c469a2007624f51999dc1fb105acdac/users/8046be534eef422cb11ec822f843a8d9/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:09.910026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4167bd-c77b-4415-a6b1-1e0a8f66db4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c469a2007624f51999dc1fb105acdac, user_id=8046be534eef422cb11ec822f843a8d9, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:09.912659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4167bd-c77b-4415-a6b1-1e0a8f66db4f admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:09.927401 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1378/2767] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:25:09 2022] PUT /identity/v3/projects/9c469a2007624f51999dc1fb105acdac/users/8046be534eef422cb11ec822f843a8d9/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:09.931454 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c8aaa-1ffc-4b37-959f-46c550b9f5ce admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:09.941515 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c8aaa-1ffc-4b37-959f-46c550b9f5ce admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:09.997284 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1f0f9-2def-4d94-a65c-61e4125dd390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:09.997896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1f0f9-2def-4d94-a65c-61e4125dd390 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:09.998175 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1f0f9-2def-4d94-a65c-61e4125dd390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:09.998447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1f0f9-2def-4d94-a65c-61e4125dd390 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:09.999148 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1f0f9-2def-4d94-a65c-61e4125dd390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:10.000744 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1f0f9-2def-4d94-a65c-61e4125dd390 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:10.010246 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1390/2768] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:25:09 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:25:10.014420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4167bd-c77b-4415-a6b1-1e0a8f66db4f admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:10.024823 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4167bd-c77b-4415-a6b1-1e0a8f66db4f admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:10.029549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea112cb-c23b-4738-a4b0-a55ba208f46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:10.030209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea112cb-c23b-4738-a4b0-a55ba208f46a admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:10.030487 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea112cb-c23b-4738-a4b0-a55ba208f46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:10.030760 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea112cb-c23b-4738-a4b0-a55ba208f46a admin admin] PATH_INFO: `/v3/projects/9c469a2007624f51999dc1fb105acdac/users/8046be534eef422cb11ec822f843a8d9/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:10.033239 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea112cb-c23b-4738-a4b0-a55ba208f46a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c469a2007624f51999dc1fb105acdac, user_id=8046be534eef422cb11ec822f843a8d9, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:10.035848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea112cb-c23b-4738-a4b0-a55ba208f46a admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:10.047185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1379/2769] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:25:10 2022] PUT /identity/v3/projects/9c469a2007624f51999dc1fb105acdac/users/8046be534eef422cb11ec822f843a8d9/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:10.058101 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3ef8e-de35-4058-aeee-4a7d80ec9b6a None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:10.058303 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3ef8e-de35-4058-aeee-4a7d80ec9b6a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:10.058624 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3ef8e-de35-4058-aeee-4a7d80ec9b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:10.085289 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-19c3ef8e-de35-4058-aeee-4a7d80ec9b6a None None] MFA Rules not processed for user `8046be534eef422cb11ec822f843a8d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:10.086479 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-19c3ef8e-de35-4058-aeee-4a7d80ec9b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:10.160014 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1391/2770] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:10 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:10.175151 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea112cb-c23b-4738-a4b0-a55ba208f46a admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:10.176992 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-5ea112cb-c23b-4738-a4b0-a55ba208f46a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:10.238012 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea112cb-c23b-4738-a4b0-a55ba208f46a admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:10.241646 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bc56d-f652-4592-9d14-982f6a44a0a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:10.242256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bc56d-f652-4592-9d14-982f6a44a0a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:10.242531 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bc56d-f652-4592-9d14-982f6a44a0a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:10.242797 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bc56d-f652-4592-9d14-982f6a44a0a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:10.251950 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bc56d-f652-4592-9d14-982f6a44a0a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:10.253398 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bc56d-f652-4592-9d14-982f6a44a0a4 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:10.263521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1380/2771] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:25:10 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:25:10.424935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3ef8e-de35-4058-aeee-4a7d80ec9b6a None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:10.436920 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3ef8e-de35-4058-aeee-4a7d80ec9b6a None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:10.441123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad955af5-5f48-4379-95ee-298558347dd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:10.441730 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad955af5-5f48-4379-95ee-298558347dd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:10.442012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad955af5-5f48-4379-95ee-298558347dd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:10.442286 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad955af5-5f48-4379-95ee-298558347dd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:10.487186 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:10.500986 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad955af5-5f48-4379-95ee-298558347dd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:10.502463 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad955af5-5f48-4379-95ee-298558347dd3 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:10.511363 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1392/2772] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:25:10 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:25:18.889188 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc46da-4baf-40cf-93f9-ed8e5ae30332 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:18.889750 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc46da-4baf-40cf-93f9-ed8e5ae30332 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:18.889944 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cc46da-4baf-40cf-93f9-ed8e5ae30332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:18.919279 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-a8cc46da-4baf-40cf-93f9-ed8e5ae30332 None None] MFA Rules not processed for user `e54e713b52e84393b018807f8153922e`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:18.920588 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a8cc46da-4baf-40cf-93f9-ed8e5ae30332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:18.926196 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:18.989093 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1381/2773] 10.222.0.45 () {58 vars in 1009 bytes} [Thu Aug 4 02:25:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:19.013012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad955af5-5f48-4379-95ee-298558347dd3 service cinder] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:19.014775 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-ad955af5-5f48-4379-95ee-298558347dd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:19.089512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad955af5-5f48-4379-95ee-298558347dd3 service cinder] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:19.094452 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3900a102-5cd1-4cc8-b528-b329ddf8711e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '155a7276efab426cb34175683c475f33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:19.095090 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900a102-5cd1-4cc8-b528-b329ddf8711e service swift] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:19.095620 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900a102-5cd1-4cc8-b528-b329ddf8711e service swift] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:19.095902 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3900a102-5cd1-4cc8-b528-b329ddf8711e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:19.104111 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900a102-5cd1-4cc8-b528-b329ddf8711e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:19.105713 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3900a102-5cd1-4cc8-b528-b329ddf8711e service swift] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:19.113866 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1393/2774] 10.222.0.45 () {60 vars in 1384 bytes} [Thu Aug 4 02:25:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 02:25:19.967191 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc46da-4baf-40cf-93f9-ed8e5ae30332 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:20.045716 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cc46da-4baf-40cf-93f9-ed8e5ae30332 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:20.052160 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fe2f5-55fc-4c45-8fb9-aa868b51d1e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:20.053423 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fe2f5-55fc-4c45-8fb9-aa868b51d1e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:20.054231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fe2f5-55fc-4c45-8fb9-aa868b51d1e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:20.054755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977fe2f5-55fc-4c45-8fb9-aa868b51d1e3 admin admin] PATH_INFO: `/v3/users/4bdfa1b58d9843ccb41bf84e246dc861` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:20.056439 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fe2f5-55fc-4c45-8fb9-aa868b51d1e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bdfa1b58d9843ccb41bf84e246dc861)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:20.058613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977fe2f5-55fc-4c45-8fb9-aa868b51d1e3 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:20.119475 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-977fe2f5-55fc-4c45-8fb9-aa868b51d1e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bdfa1b58d9843ccb41bf84e246dc861'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:20.120285 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-977fe2f5-55fc-4c45-8fb9-aa868b51d1e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bdfa1b58d9843ccb41bf84e246dc861'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:20.121343 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-977fe2f5-55fc-4c45-8fb9-aa868b51d1e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bdfa1b58d9843ccb41bf84e246dc861'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:20.137822 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-977fe2f5-55fc-4c45-8fb9-aa868b51d1e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bdfa1b58d9843ccb41bf84e246dc861'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:20.146807 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-977fe2f5-55fc-4c45-8fb9-aa868b51d1e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bdfa1b58d9843ccb41bf84e246dc861'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:20.164224 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1382/2775] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:25:19 2022] DELETE /identity/v3/users/4bdfa1b58d9843ccb41bf84e246dc861 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:20.168877 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134f4a6-1caf-4043-b1de-c1467c7f7440 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:20.169857 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134f4a6-1caf-4043-b1de-c1467c7f7440 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:20.170361 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134f4a6-1caf-4043-b1de-c1467c7f7440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:20.195513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-4134f4a6-1caf-4043-b1de-c1467c7f7440 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:20.196883 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4134f4a6-1caf-4043-b1de-c1467c7f7440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:20.273455 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1394/2776] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:20.283874 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fe2f5-55fc-4c45-8fb9-aa868b51d1e3 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:20.285367 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-977fe2f5-55fc-4c45-8fb9-aa868b51d1e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:20.330767 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:20.387393 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977fe2f5-55fc-4c45-8fb9-aa868b51d1e3 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:20.391722 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc88bb-ec6b-4cc8-99b3-1491e637d370 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:20.392285 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc88bb-ec6b-4cc8-99b3-1491e637d370 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:20.392745 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc88bb-ec6b-4cc8-99b3-1491e637d370 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:20.393116 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc88bb-ec6b-4cc8-99b3-1491e637d370 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:20.403033 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc88bb-ec6b-4cc8-99b3-1491e637d370 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:20.404564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc88bb-ec6b-4cc8-99b3-1491e637d370 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:20.417550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1383/2777] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 04 02:25:20.777824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134f4a6-1caf-4043-b1de-c1467c7f7440 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:20.779973 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4134f4a6-1caf-4043-b1de-c1467c7f7440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:20.808897 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134f4a6-1caf-4043-b1de-c1467c7f7440 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:20.824174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ee6fd-7784-409e-9a8c-a7569cc542c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:20.825144 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ee6fd-7784-409e-9a8c-a7569cc542c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:20.825626 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ee6fd-7784-409e-9a8c-a7569cc542c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:20.825903 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ee6fd-7784-409e-9a8c-a7569cc542c4 admin admin] PATH_INFO: `/v3/projects/c21917f9c96d49eda52b24d71a6c7d82` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:20.827084 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ee6fd-7784-409e-9a8c-a7569cc542c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c21917f9c96d49eda52b24d71a6c7d82)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:20.829700 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ee6fd-7784-409e-9a8c-a7569cc542c4 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:20.910394 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-427ee6fd-7784-409e-9a8c-a7569cc542c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c21917f9c96d49eda52b24d71a6c7d82', 'request_id': 'req-427ee6fd-7784-409e-9a8c-a7569cc542c4'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:20.922521 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-427ee6fd-7784-409e-9a8c-a7569cc542c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c21917f9c96d49eda52b24d71a6c7d82', 'request_id': 'req-427ee6fd-7784-409e-9a8c-a7569cc542c4'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:20.932746 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-427ee6fd-7784-409e-9a8c-a7569cc542c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c21917f9c96d49eda52b24d71a6c7d82', 'request_id': 'req-427ee6fd-7784-409e-9a8c-a7569cc542c4'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:20.942497 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:20.947156 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-427ee6fd-7784-409e-9a8c-a7569cc542c4 admin admin] The token cache is being invalidate because project c21917f9c96d49eda52b24d71a6c7d82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:25:20.948326 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-427ee6fd-7784-409e-9a8c-a7569cc542c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:20.949536 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-427ee6fd-7784-409e-9a8c-a7569cc542c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:20.951323 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1395/2778] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:25:20 2022] DELETE /identity/v3/projects/c21917f9c96d49eda52b24d71a6c7d82 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:20.956202 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc88bb-ec6b-4cc8-99b3-1491e637d370 service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:20.968760 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc88bb-ec6b-4cc8-99b3-1491e637d370 service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:21.036049 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab665d-d963-4904-9d03-7e1f0d130eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:21.036729 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab665d-d963-4904-9d03-7e1f0d130eb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:21.037035 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab665d-d963-4904-9d03-7e1f0d130eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:21.037316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab665d-d963-4904-9d03-7e1f0d130eb1 admin admin] PATH_INFO: `/v3/users/8046be534eef422cb11ec822f843a8d9` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:21.038512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab665d-d963-4904-9d03-7e1f0d130eb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8046be534eef422cb11ec822f843a8d9)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:21.040452 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deab665d-d963-4904-9d03-7e1f0d130eb1 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:21.093925 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-deab665d-d963-4904-9d03-7e1f0d130eb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8046be534eef422cb11ec822f843a8d9'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:21.094596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-deab665d-d963-4904-9d03-7e1f0d130eb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8046be534eef422cb11ec822f843a8d9'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:21.096068 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-deab665d-d963-4904-9d03-7e1f0d130eb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8046be534eef422cb11ec822f843a8d9'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:21.110256 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-deab665d-d963-4904-9d03-7e1f0d130eb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8046be534eef422cb11ec822f843a8d9'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:21.118141 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-deab665d-d963-4904-9d03-7e1f0d130eb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8046be534eef422cb11ec822f843a8d9'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:21.135649 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1384/2779] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:25:20 2022] DELETE /identity/v3/users/8046be534eef422cb11ec822f843a8d9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:21.486776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ee6fd-7784-409e-9a8c-a7569cc542c4 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:21.488424 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-427ee6fd-7784-409e-9a8c-a7569cc542c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:21.534220 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ee6fd-7784-409e-9a8c-a7569cc542c4 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:21.611135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aab88bf-bd8b-4a2f-98dd-446020469945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:21.612212 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab88bf-bd8b-4a2f-98dd-446020469945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:21.612642 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab88bf-bd8b-4a2f-98dd-446020469945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:21.612913 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aab88bf-bd8b-4a2f-98dd-446020469945 admin admin] PATH_INFO: `/v3/projects/9c469a2007624f51999dc1fb105acdac` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:21.614283 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab88bf-bd8b-4a2f-98dd-446020469945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c469a2007624f51999dc1fb105acdac)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:21.616178 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aab88bf-bd8b-4a2f-98dd-446020469945 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:21.680050 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3aab88bf-bd8b-4a2f-98dd-446020469945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c469a2007624f51999dc1fb105acdac', 'request_id': 'req-3aab88bf-bd8b-4a2f-98dd-446020469945'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:21.690832 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3aab88bf-bd8b-4a2f-98dd-446020469945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c469a2007624f51999dc1fb105acdac', 'request_id': 'req-3aab88bf-bd8b-4a2f-98dd-446020469945'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:21.697590 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3aab88bf-bd8b-4a2f-98dd-446020469945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c469a2007624f51999dc1fb105acdac', 'request_id': 'req-3aab88bf-bd8b-4a2f-98dd-446020469945'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:21.710439 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3aab88bf-bd8b-4a2f-98dd-446020469945 admin admin] The token cache is being invalidate because project 9c469a2007624f51999dc1fb105acdac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:25:21.710738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3aab88bf-bd8b-4a2f-98dd-446020469945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:21.711427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3aab88bf-bd8b-4a2f-98dd-446020469945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:21.713197 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1396/2780] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:25:21 2022] DELETE /identity/v3/projects/9c469a2007624f51999dc1fb105acdac => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:21.724683 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a55342-804a-45a5-9e66-aa1c909d497b None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:21.724683 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a55342-804a-45a5-9e66-aa1c909d497b None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:21.724683 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a55342-804a-45a5-9e66-aa1c909d497b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:21.741472 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-10a55342-804a-45a5-9e66-aa1c909d497b None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:21.742671 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-10a55342-804a-45a5-9e66-aa1c909d497b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:21.773310 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:21.834739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1385/2781] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:21.844047 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc78fa-ebbf-4c5e-9887-1929d579a7cf None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:21.844334 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc78fa-ebbf-4c5e-9887-1929d579a7cf None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:21.845228 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc78fa-ebbf-4c5e-9887-1929d579a7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:21.878539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-a8fc78fa-ebbf-4c5e-9887-1929d579a7cf None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:21.879918 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a8fc78fa-ebbf-4c5e-9887-1929d579a7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:21.901068 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1397/2782] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:21.901458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a55342-804a-45a5-9e66-aa1c909d497b None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:21.916829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a55342-804a-45a5-9e66-aa1c909d497b None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:21.920682 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcba90d-952d-419e-aefa-e4b74ff07255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:21.921316 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcba90d-952d-419e-aefa-e4b74ff07255 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:21.921592 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcba90d-952d-419e-aefa-e4b74ff07255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:21.921861 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcba90d-952d-419e-aefa-e4b74ff07255 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:21.922560 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcba90d-952d-419e-aefa-e4b74ff07255 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:25:21.923419 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcba90d-952d-419e-aefa-e4b74ff07255 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:21.925047 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcba90d-952d-419e-aefa-e4b74ff07255 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:21.937469 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1386/2783] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:25:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:25:21.942043 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc78fa-ebbf-4c5e-9887-1929d579a7cf None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:21.952120 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc78fa-ebbf-4c5e-9887-1929d579a7cf None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:21.957428 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37b8d0-0ab5-4a8c-b93c-f2f9b61558e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:21.957664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37b8d0-0ab5-4a8c-b93c-f2f9b61558e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:21.957905 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37b8d0-0ab5-4a8c-b93c-f2f9b61558e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:21.958198 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37b8d0-0ab5-4a8c-b93c-f2f9b61558e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:21.958860 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37b8d0-0ab5-4a8c-b93c-f2f9b61558e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:21.960757 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37b8d0-0ab5-4a8c-b93c-f2f9b61558e1 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:21.997183 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1398/2784] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:25:21 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:25:22.001117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcba90d-952d-419e-aefa-e4b74ff07255 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:22.013551 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcba90d-952d-419e-aefa-e4b74ff07255 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:22.079704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a535bc-742b-4f08-a25e-a350ba28ee54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:22.080441 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a535bc-742b-4f08-a25e-a350ba28ee54 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:22.080719 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a535bc-742b-4f08-a25e-a350ba28ee54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:22.080979 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a535bc-742b-4f08-a25e-a350ba28ee54 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:22.081615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a535bc-742b-4f08-a25e-a350ba28ee54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:22.083645 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a535bc-742b-4f08-a25e-a350ba28ee54 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:22.112670 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1387/2785] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:25:22 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:25:22.115928 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37b8d0-0ab5-4a8c-b93c-f2f9b61558e1 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:22.128169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37b8d0-0ab5-4a8c-b93c-f2f9b61558e1 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:22.134588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f536de-fbc1-429f-acba-455b4883ccba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:22.135243 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f536de-fbc1-429f-acba-455b4883ccba admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:22.135513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f536de-fbc1-429f-acba-455b4883ccba admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:22.135793 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f536de-fbc1-429f-acba-455b4883ccba admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:22.136512 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f536de-fbc1-429f-acba-455b4883ccba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:22.138095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f536de-fbc1-429f-acba-455b4883ccba admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:22.160749 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1399/2786] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:25:22.162872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a535bc-742b-4f08-a25e-a350ba28ee54 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:22.178019 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a535bc-742b-4f08-a25e-a350ba28ee54 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:22.183128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aa839c-6a4e-4adb-a410-a6d9c4e61d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:22.183747 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa839c-6a4e-4adb-a410-a6d9c4e61d64 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:22.184084 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa839c-6a4e-4adb-a410-a6d9c4e61d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:22.184353 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aa839c-6a4e-4adb-a410-a6d9c4e61d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:22.185105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa839c-6a4e-4adb-a410-a6d9c4e61d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:22.186701 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aa839c-6a4e-4adb-a410-a6d9c4e61d64 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:22.200420 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1388/2787] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:25:22.204324 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f536de-fbc1-429f-acba-455b4883ccba admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:22.217040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f536de-fbc1-429f-acba-455b4883ccba admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:22.222881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93022a58-edf2-406c-b815-c0319f25e16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:22.223670 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93022a58-edf2-406c-b815-c0319f25e16a admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:22.223944 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93022a58-edf2-406c-b815-c0319f25e16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:22.224210 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93022a58-edf2-406c-b815-c0319f25e16a admin admin] PATH_INFO: `/v3/projects/f7a3daffe7e94e8496e8af76b6629d28/users/29983b12900840c08c2dd1da8e57ac64/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:22.241264 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93022a58-edf2-406c-b815-c0319f25e16a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7a3daffe7e94e8496e8af76b6629d28, user_id=29983b12900840c08c2dd1da8e57ac64, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:22.243892 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93022a58-edf2-406c-b815-c0319f25e16a admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:22.272388 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1400/2788] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:25:22 2022] PUT /identity/v3/projects/f7a3daffe7e94e8496e8af76b6629d28/users/29983b12900840c08c2dd1da8e57ac64/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:22.274923 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d8938-b8c9-4500-9d59-6f23c63341ff None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:22.275477 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d8938-b8c9-4500-9d59-6f23c63341ff None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:22.276396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d8938-b8c9-4500-9d59-6f23c63341ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:22.316431 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-268d8938-b8c9-4500-9d59-6f23c63341ff None None] MFA Rules not processed for user `29983b12900840c08c2dd1da8e57ac64`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:22.317773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-268d8938-b8c9-4500-9d59-6f23c63341ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:22.400346 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1389/2789] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:22.405080 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93022a58-edf2-406c-b815-c0319f25e16a admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:22.442842 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:22.496507 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93022a58-edf2-406c-b815-c0319f25e16a admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:22.503436 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00422e4c-748b-4e32-892b-69385335aaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:22.504827 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00422e4c-748b-4e32-892b-69385335aaed admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:22.505189 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00422e4c-748b-4e32-892b-69385335aaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:22.505399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00422e4c-748b-4e32-892b-69385335aaed admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:22.506227 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00422e4c-748b-4e32-892b-69385335aaed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:22.508384 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00422e4c-748b-4e32-892b-69385335aaed admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:22.552883 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1401/2790] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:25:22 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:25:22.553848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d8938-b8c9-4500-9d59-6f23c63341ff None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:22.570243 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d8938-b8c9-4500-9d59-6f23c63341ff None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:22.606298 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:22.657795 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85032aca-0746-4a1d-bbfe-c12cb169b14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:22.659098 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85032aca-0746-4a1d-bbfe-c12cb169b14b admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:22.659596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85032aca-0746-4a1d-bbfe-c12cb169b14b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:22.660131 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85032aca-0746-4a1d-bbfe-c12cb169b14b admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:22.661146 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85032aca-0746-4a1d-bbfe-c12cb169b14b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:22.663753 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85032aca-0746-4a1d-bbfe-c12cb169b14b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:22.692405 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1390/2791] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:25:22 2022] POST /identity/v3/users => generated 290 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:25:22.696998 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00422e4c-748b-4e32-892b-69385335aaed admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:22.710759 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00422e4c-748b-4e32-892b-69385335aaed admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:22.716224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ba7f4-d6e3-4270-82c8-79119ff31c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:22.716871 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ba7f4-d6e3-4270-82c8-79119ff31c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:22.717154 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ba7f4-d6e3-4270-82c8-79119ff31c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:22.717438 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939ba7f4-d6e3-4270-82c8-79119ff31c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:22.718143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939ba7f4-d6e3-4270-82c8-79119ff31c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:22.719821 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939ba7f4-d6e3-4270-82c8-79119ff31c85 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:22.731790 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1402/2792] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:25:22.736326 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85032aca-0746-4a1d-bbfe-c12cb169b14b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:22.750157 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85032aca-0746-4a1d-bbfe-c12cb169b14b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:22.756786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28013eea-4bf2-4100-8228-81635811ccef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:22.757659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28013eea-4bf2-4100-8228-81635811ccef admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:22.758036 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28013eea-4bf2-4100-8228-81635811ccef admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:22.758313 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28013eea-4bf2-4100-8228-81635811ccef admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:22.759174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28013eea-4bf2-4100-8228-81635811ccef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:22.761422 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28013eea-4bf2-4100-8228-81635811ccef admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:22.777339 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1391/2793] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:25:22.778776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ba7f4-d6e3-4270-82c8-79119ff31c85 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:22.792153 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939ba7f4-d6e3-4270-82c8-79119ff31c85 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:22.800076 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ff443-c82a-4e37-a07b-77152b1f3ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:22.800824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ff443-c82a-4e37-a07b-77152b1f3ae9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:22.801026 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ff443-c82a-4e37-a07b-77152b1f3ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:22.801298 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ff443-c82a-4e37-a07b-77152b1f3ae9 admin admin] PATH_INFO: `/v3/projects/d05ae6f048754adb898ced7701f4b397/users/c314978a3bdb4026a6de3a608037da5e/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:22.823256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ff443-c82a-4e37-a07b-77152b1f3ae9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d05ae6f048754adb898ced7701f4b397, user_id=c314978a3bdb4026a6de3a608037da5e, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:22.826245 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ff443-c82a-4e37-a07b-77152b1f3ae9 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:22.865104 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce480f-b324-403c-bce9-71e16248724b None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:22.865462 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1403/2794] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:25:22 2022] PUT /identity/v3/projects/d05ae6f048754adb898ced7701f4b397/users/c314978a3bdb4026a6de3a608037da5e/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:22.865767 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce480f-b324-403c-bce9-71e16248724b None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:22.865999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ce480f-b324-403c-bce9-71e16248724b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:22.904976 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-a3ce480f-b324-403c-bce9-71e16248724b None None] MFA Rules not processed for user `c314978a3bdb4026a6de3a608037da5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:22.906542 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a3ce480f-b324-403c-bce9-71e16248724b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:23.010685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1392/2795] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:23.015571 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ff443-c82a-4e37-a07b-77152b1f3ae9 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:23.090048 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ff443-c82a-4e37-a07b-77152b1f3ae9 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:23.094971 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc55b5-a372-4763-ac9b-9cecd0177b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:23.095844 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc55b5-a372-4763-ac9b-9cecd0177b25 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:23.096121 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc55b5-a372-4763-ac9b-9cecd0177b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:23.096395 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fc55b5-a372-4763-ac9b-9cecd0177b25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:23.097228 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc55b5-a372-4763-ac9b-9cecd0177b25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:23.099238 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fc55b5-a372-4763-ac9b-9cecd0177b25 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:23.132071 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1404/2796] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:25:23 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:25:23.136444 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce480f-b324-403c-bce9-71e16248724b None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:23.156325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ce480f-b324-403c-bce9-71e16248724b None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:23.242079 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735f68c-563a-486d-81ce-0b07ecda34a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:23.242858 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735f68c-563a-486d-81ce-0b07ecda34a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:23.243209 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735f68c-563a-486d-81ce-0b07ecda34a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:23.243587 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735f68c-563a-486d-81ce-0b07ecda34a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:23.244347 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735f68c-563a-486d-81ce-0b07ecda34a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:23.246667 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735f68c-563a-486d-81ce-0b07ecda34a1 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:23.287919 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1393/2797] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:25:23 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:25:23.289218 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc55b5-a372-4763-ac9b-9cecd0177b25 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:23.303313 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fc55b5-a372-4763-ac9b-9cecd0177b25 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:23.308569 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc8ad0-e739-43fd-ae42-b872f3c2bee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:23.309395 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc8ad0-e739-43fd-ae42-b872f3c2bee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:23.309681 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc8ad0-e739-43fd-ae42-b872f3c2bee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:23.309950 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc8ad0-e739-43fd-ae42-b872f3c2bee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:23.310646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc8ad0-e739-43fd-ae42-b872f3c2bee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:23.312213 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc8ad0-e739-43fd-ae42-b872f3c2bee0 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:23.328468 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1405/2798] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:25:23.331462 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735f68c-563a-486d-81ce-0b07ecda34a1 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:23.345127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735f68c-563a-486d-81ce-0b07ecda34a1 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:23.349956 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebdf304-7e53-49fc-b87c-0e16e72d1899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:23.352932 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdf304-7e53-49fc-b87c-0e16e72d1899 admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:23.353213 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdf304-7e53-49fc-b87c-0e16e72d1899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:23.353483 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebdf304-7e53-49fc-b87c-0e16e72d1899 admin admin] PATH_INFO: `/v3/projects/a9dbd2d6dd1d4538910b5ede15bc3ca2/users/ac1ad8b3e7474816b8750624ae07b76f/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:23.372512 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebdf304-7e53-49fc-b87c-0e16e72d1899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9dbd2d6dd1d4538910b5ede15bc3ca2, user_id=ac1ad8b3e7474816b8750624ae07b76f, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:23.375190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebdf304-7e53-49fc-b87c-0e16e72d1899 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:23.387333 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:23.397491 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1394/2799] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:25:23 2022] PUT /identity/v3/projects/a9dbd2d6dd1d4538910b5ede15bc3ca2/users/ac1ad8b3e7474816b8750624ae07b76f/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:23.398833 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc8ad0-e739-43fd-ae42-b872f3c2bee0 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:23.410247 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc8ad0-e739-43fd-ae42-b872f3c2bee0 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:23.468965 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:23.481470 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d338de20-c5f7-484a-8a9e-dffdd1022502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:23.482208 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338de20-c5f7-484a-8a9e-dffdd1022502 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:23.482764 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338de20-c5f7-484a-8a9e-dffdd1022502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:23.483040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d338de20-c5f7-484a-8a9e-dffdd1022502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:23.483881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338de20-c5f7-484a-8a9e-dffdd1022502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:23.486443 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d338de20-c5f7-484a-8a9e-dffdd1022502 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:23.501435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebdf304-7e53-49fc-b87c-0e16e72d1899 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:23.501911 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1406/2800] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:25:23.517614 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebdf304-7e53-49fc-b87c-0e16e72d1899 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:23.531598 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d88e0-5e47-455a-a905-be2e255eb66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:23.532853 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d88e0-5e47-455a-a905-be2e255eb66b admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:23.533126 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d88e0-5e47-455a-a905-be2e255eb66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:23.533395 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d88e0-5e47-455a-a905-be2e255eb66b admin admin] PATH_INFO: `/v3/projects/a9dbd2d6dd1d4538910b5ede15bc3ca2/users/ac1ad8b3e7474816b8750624ae07b76f/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:23.535660 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d88e0-5e47-455a-a905-be2e255eb66b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9dbd2d6dd1d4538910b5ede15bc3ca2, user_id=ac1ad8b3e7474816b8750624ae07b76f, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:23.538592 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d88e0-5e47-455a-a905-be2e255eb66b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:23.550978 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1395/2801] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:25:23 2022] PUT /identity/v3/projects/a9dbd2d6dd1d4538910b5ede15bc3ca2/users/ac1ad8b3e7474816b8750624ae07b76f/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:23.621899 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a592c-7cd5-4855-9fd6-69423920766f None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:23.622254 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a592c-7cd5-4855-9fd6-69423920766f None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:23.622448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a592c-7cd5-4855-9fd6-69423920766f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:23.656565 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-af0a592c-7cd5-4855-9fd6-69423920766f None None] MFA Rules not processed for user `ac1ad8b3e7474816b8750624ae07b76f`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:23.657994 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-af0a592c-7cd5-4855-9fd6-69423920766f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:23.754919 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1407/2802] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:23 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:23.767262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d88e0-5e47-455a-a905-be2e255eb66b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:23.769069 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-051d88e0-5e47-455a-a905-be2e255eb66b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:23.877549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d88e0-5e47-455a-a905-be2e255eb66b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:23.881627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df20fb6-a099-452c-9453-87dd9543749f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:23.882400 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df20fb6-a099-452c-9453-87dd9543749f service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:23.882769 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df20fb6-a099-452c-9453-87dd9543749f service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:23.883126 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df20fb6-a099-452c-9453-87dd9543749f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:23.900778 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df20fb6-a099-452c-9453-87dd9543749f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:23.902494 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df20fb6-a099-452c-9453-87dd9543749f service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:23.915564 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1396/2803] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:25:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:25:23.966606 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a592c-7cd5-4855-9fd6-69423920766f None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:23.968077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-af0a592c-7cd5-4855-9fd6-69423920766f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:23.989332 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a592c-7cd5-4855-9fd6-69423920766f None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:23.993617 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7cbde-a766-46b1-bad0-406e76f86077 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:23.994337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7cbde-a766-46b1-bad0-406e76f86077 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:23.994686 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7cbde-a766-46b1-bad0-406e76f86077 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:23.995025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7cbde-a766-46b1-bad0-406e76f86077 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:24.066990 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7cbde-a766-46b1-bad0-406e76f86077 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:24.068599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7cbde-a766-46b1-bad0-406e76f86077 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:24.081788 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1408/2804] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:25:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:25:27.174338 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df20fb6-a099-452c-9453-87dd9543749f service cinder] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:27.249063 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df20fb6-a099-452c-9453-87dd9543749f service cinder] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:27.254598 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a6dd0-14e0-498a-bbf8-51115cf58f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:27.255388 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a6dd0-14e0-498a-bbf8-51115cf58f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:27.255735 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a6dd0-14e0-498a-bbf8-51115cf58f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:27.256079 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19a6dd0-14e0-498a-bbf8-51115cf58f7e admin admin] PATH_INFO: `/v3/users/29983b12900840c08c2dd1da8e57ac64` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:27.257489 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a6dd0-14e0-498a-bbf8-51115cf58f7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=29983b12900840c08c2dd1da8e57ac64)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:27.259557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19a6dd0-14e0-498a-bbf8-51115cf58f7e admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:27.320172 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e19a6dd0-14e0-498a-bbf8-51115cf58f7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29983b12900840c08c2dd1da8e57ac64'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:27.320885 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e19a6dd0-14e0-498a-bbf8-51115cf58f7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29983b12900840c08c2dd1da8e57ac64'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:27.321498 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e19a6dd0-14e0-498a-bbf8-51115cf58f7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29983b12900840c08c2dd1da8e57ac64'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:27.339413 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e19a6dd0-14e0-498a-bbf8-51115cf58f7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29983b12900840c08c2dd1da8e57ac64'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:27.346680 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:27.349106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e19a6dd0-14e0-498a-bbf8-51115cf58f7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29983b12900840c08c2dd1da8e57ac64'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:27.380658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1397/2805] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:25:27 2022] DELETE /identity/v3/users/29983b12900840c08c2dd1da8e57ac64 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:27.384970 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946393e-2531-486e-b19d-86da18e489e3 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:27.385543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946393e-2531-486e-b19d-86da18e489e3 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:27.385543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946393e-2531-486e-b19d-86da18e489e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:27.405762 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-4946393e-2531-486e-b19d-86da18e489e3 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:27.407612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4946393e-2531-486e-b19d-86da18e489e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:27.508761 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1409/2806] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:27.519205 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a6dd0-14e0-498a-bbf8-51115cf58f7e admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:27.520676 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e19a6dd0-14e0-498a-bbf8-51115cf58f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:27.591419 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19a6dd0-14e0-498a-bbf8-51115cf58f7e admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:27.598236 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf1005-d138-4744-a5bd-4b6c8a23ea58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:27.598870 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf1005-d138-4744-a5bd-4b6c8a23ea58 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:27.599143 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf1005-d138-4744-a5bd-4b6c8a23ea58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:27.599412 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf1005-d138-4744-a5bd-4b6c8a23ea58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:27.612042 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf1005-d138-4744-a5bd-4b6c8a23ea58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:27.613518 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf1005-d138-4744-a5bd-4b6c8a23ea58 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:27.623926 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1398/2807] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:25:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:25:28.009712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4946393e-2531-486e-b19d-86da18e489e3 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:28.012097 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4946393e-2531-486e-b19d-86da18e489e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:28.025770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4946393e-2531-486e-b19d-86da18e489e3 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:28.064152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa939e-98c0-4a65-be27-00e32c7ee56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:28.073986 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa939e-98c0-4a65-be27-00e32c7ee56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:28.074447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa939e-98c0-4a65-be27-00e32c7ee56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:28.074855 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa939e-98c0-4a65-be27-00e32c7ee56d admin admin] PATH_INFO: `/v3/projects/f7a3daffe7e94e8496e8af76b6629d28` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:28.077872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa939e-98c0-4a65-be27-00e32c7ee56d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7a3daffe7e94e8496e8af76b6629d28)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:28.080274 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa939e-98c0-4a65-be27-00e32c7ee56d admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:28.089777 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:28.142699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4afa939e-98c0-4a65-be27-00e32c7ee56d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7a3daffe7e94e8496e8af76b6629d28', 'request_id': 'req-4afa939e-98c0-4a65-be27-00e32c7ee56d'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:28.151946 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4afa939e-98c0-4a65-be27-00e32c7ee56d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7a3daffe7e94e8496e8af76b6629d28', 'request_id': 'req-4afa939e-98c0-4a65-be27-00e32c7ee56d'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:28.160189 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4afa939e-98c0-4a65-be27-00e32c7ee56d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7a3daffe7e94e8496e8af76b6629d28', 'request_id': 'req-4afa939e-98c0-4a65-be27-00e32c7ee56d'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:28.174755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4afa939e-98c0-4a65-be27-00e32c7ee56d admin admin] The token cache is being invalidate because project f7a3daffe7e94e8496e8af76b6629d28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:25:28.175399 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4afa939e-98c0-4a65-be27-00e32c7ee56d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:28.176786 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-4afa939e-98c0-4a65-be27-00e32c7ee56d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:28.178960 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1410/2808] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:25:28 2022] DELETE /identity/v3/projects/f7a3daffe7e94e8496e8af76b6629d28 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:28.182971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf1005-d138-4744-a5bd-4b6c8a23ea58 service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:28.197117 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf1005-d138-4744-a5bd-4b6c8a23ea58 service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:28.289981 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903df1a-ff55-4a5d-812e-510a9d033a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:28.291090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903df1a-ff55-4a5d-812e-510a9d033a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:28.291627 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903df1a-ff55-4a5d-812e-510a9d033a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:28.292143 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903df1a-ff55-4a5d-812e-510a9d033a5b admin admin] PATH_INFO: `/v3/users/c314978a3bdb4026a6de3a608037da5e` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:28.294523 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903df1a-ff55-4a5d-812e-510a9d033a5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c314978a3bdb4026a6de3a608037da5e)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:28.296648 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903df1a-ff55-4a5d-812e-510a9d033a5b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:28.357169 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-6903df1a-ff55-4a5d-812e-510a9d033a5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c314978a3bdb4026a6de3a608037da5e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:28.357852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-6903df1a-ff55-4a5d-812e-510a9d033a5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c314978a3bdb4026a6de3a608037da5e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:28.358696 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-6903df1a-ff55-4a5d-812e-510a9d033a5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c314978a3bdb4026a6de3a608037da5e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:28.376521 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-6903df1a-ff55-4a5d-812e-510a9d033a5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c314978a3bdb4026a6de3a608037da5e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:28.387195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-6903df1a-ff55-4a5d-812e-510a9d033a5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c314978a3bdb4026a6de3a608037da5e'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:28.405969 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1399/2809] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:25:28 2022] DELETE /identity/v3/users/c314978a3bdb4026a6de3a608037da5e => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:28.746934 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa939e-98c0-4a65-be27-00e32c7ee56d admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:28.749376 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4afa939e-98c0-4a65-be27-00e32c7ee56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:28.772483 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa939e-98c0-4a65-be27-00e32c7ee56d admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:28.864239 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263889a-0566-45b4-bfc6-f8ff972728cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:28.865095 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263889a-0566-45b4-bfc6-f8ff972728cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:28.865477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263889a-0566-45b4-bfc6-f8ff972728cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:28.865858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a263889a-0566-45b4-bfc6-f8ff972728cb admin admin] PATH_INFO: `/v3/projects/d05ae6f048754adb898ced7701f4b397` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:28.867261 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263889a-0566-45b4-bfc6-f8ff972728cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d05ae6f048754adb898ced7701f4b397)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:28.869306 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a263889a-0566-45b4-bfc6-f8ff972728cb admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:28.925734 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a263889a-0566-45b4-bfc6-f8ff972728cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd05ae6f048754adb898ced7701f4b397', 'request_id': 'req-a263889a-0566-45b4-bfc6-f8ff972728cb'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:28.935809 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a263889a-0566-45b4-bfc6-f8ff972728cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd05ae6f048754adb898ced7701f4b397', 'request_id': 'req-a263889a-0566-45b4-bfc6-f8ff972728cb'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:28.944579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a263889a-0566-45b4-bfc6-f8ff972728cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd05ae6f048754adb898ced7701f4b397', 'request_id': 'req-a263889a-0566-45b4-bfc6-f8ff972728cb'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:28.960409 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a263889a-0566-45b4-bfc6-f8ff972728cb admin admin] The token cache is being invalidate because project d05ae6f048754adb898ced7701f4b397 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:25:28.960748 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a263889a-0566-45b4-bfc6-f8ff972728cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:28.961496 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-a263889a-0566-45b4-bfc6-f8ff972728cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:28.966475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1411/2810] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:25:28 2022] DELETE /identity/v3/projects/d05ae6f048754adb898ced7701f4b397 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:28.968463 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903df1a-ff55-4a5d-812e-510a9d033a5b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:28.970816 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-6903df1a-ff55-4a5d-812e-510a9d033a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:28.985970 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903df1a-ff55-4a5d-812e-510a9d033a5b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:28.995318 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:29.058886 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d582ea1f-4823-4f94-8503-828a431141bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:29.059692 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582ea1f-4823-4f94-8503-828a431141bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:29.060068 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582ea1f-4823-4f94-8503-828a431141bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:29.060436 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582ea1f-4823-4f94-8503-828a431141bf admin admin] PATH_INFO: `/v3/users/ac1ad8b3e7474816b8750624ae07b76f` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:29.061731 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582ea1f-4823-4f94-8503-828a431141bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac1ad8b3e7474816b8750624ae07b76f)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:29.063862 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d582ea1f-4823-4f94-8503-828a431141bf admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:29.119829 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d582ea1f-4823-4f94-8503-828a431141bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac1ad8b3e7474816b8750624ae07b76f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:29.120493 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d582ea1f-4823-4f94-8503-828a431141bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac1ad8b3e7474816b8750624ae07b76f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:29.121137 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d582ea1f-4823-4f94-8503-828a431141bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac1ad8b3e7474816b8750624ae07b76f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:29.141013 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d582ea1f-4823-4f94-8503-828a431141bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac1ad8b3e7474816b8750624ae07b76f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:29.152319 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-d582ea1f-4823-4f94-8503-828a431141bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac1ad8b3e7474816b8750624ae07b76f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:29.169971 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1400/2811] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:25:28 2022] DELETE /identity/v3/users/ac1ad8b3e7474816b8750624ae07b76f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:29.528558 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263889a-0566-45b4-bfc6-f8ff972728cb admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:29.530094 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-a263889a-0566-45b4-bfc6-f8ff972728cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:29.552392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a263889a-0566-45b4-bfc6-f8ff972728cb admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:29.583636 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:29.633274 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13a78e-814d-46b1-b3bd-f4532897c8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:29.634083 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13a78e-814d-46b1-b3bd-f4532897c8a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:29.634370 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13a78e-814d-46b1-b3bd-f4532897c8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:29.634641 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13a78e-814d-46b1-b3bd-f4532897c8a7 admin admin] PATH_INFO: `/v3/projects/a9dbd2d6dd1d4538910b5ede15bc3ca2` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:29.635838 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13a78e-814d-46b1-b3bd-f4532897c8a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9dbd2d6dd1d4538910b5ede15bc3ca2)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:29.637927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13a78e-814d-46b1-b3bd-f4532897c8a7 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:29.694253 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fe13a78e-814d-46b1-b3bd-f4532897c8a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9dbd2d6dd1d4538910b5ede15bc3ca2', 'request_id': 'req-fe13a78e-814d-46b1-b3bd-f4532897c8a7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:29.703415 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fe13a78e-814d-46b1-b3bd-f4532897c8a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9dbd2d6dd1d4538910b5ede15bc3ca2', 'request_id': 'req-fe13a78e-814d-46b1-b3bd-f4532897c8a7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:29.712955 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fe13a78e-814d-46b1-b3bd-f4532897c8a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9dbd2d6dd1d4538910b5ede15bc3ca2', 'request_id': 'req-fe13a78e-814d-46b1-b3bd-f4532897c8a7'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:29.727152 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fe13a78e-814d-46b1-b3bd-f4532897c8a7 admin admin] The token cache is being invalidate because project a9dbd2d6dd1d4538910b5ede15bc3ca2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:25:29.727448 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fe13a78e-814d-46b1-b3bd-f4532897c8a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:29.728143 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fe13a78e-814d-46b1-b3bd-f4532897c8a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:29.734312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1412/2812] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:25:29 2022] DELETE /identity/v3/projects/a9dbd2d6dd1d4538910b5ede15bc3ca2 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:29.741088 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32435e-d25a-4761-97d4-81f4ab954565 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:29.742710 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32435e-d25a-4761-97d4-81f4ab954565 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:29.743654 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32435e-d25a-4761-97d4-81f4ab954565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:29.778501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-dc32435e-d25a-4761-97d4-81f4ab954565 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:29.780380 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-dc32435e-d25a-4761-97d4-81f4ab954565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:29.854497 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1401/2813] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:29.862828 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067098d-5b9f-44d8-a6d2-e6557263f184 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:29.863336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067098d-5b9f-44d8-a6d2-e6557263f184 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:29.863422 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6067098d-5b9f-44d8-a6d2-e6557263f184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:29.885877 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-6067098d-5b9f-44d8-a6d2-e6557263f184 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:29.887113 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-6067098d-5b9f-44d8-a6d2-e6557263f184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:29.896608 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1413/2814] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:29.900327 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32435e-d25a-4761-97d4-81f4ab954565 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:29.913730 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32435e-d25a-4761-97d4-81f4ab954565 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:29.919327 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815715d-8b93-4138-b0ce-54284054f111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:29.920089 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815715d-8b93-4138-b0ce-54284054f111 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:29.920464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815715d-8b93-4138-b0ce-54284054f111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:29.920870 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815715d-8b93-4138-b0ce-54284054f111 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:29.921658 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815715d-8b93-4138-b0ce-54284054f111 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:25:29.922026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815715d-8b93-4138-b0ce-54284054f111 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:29.923752 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815715d-8b93-4138-b0ce-54284054f111 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:29.940670 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1402/2815] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:25:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:25:29.942678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067098d-5b9f-44d8-a6d2-e6557263f184 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:29.952406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6067098d-5b9f-44d8-a6d2-e6557263f184 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:29.958018 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431aabd1-4a38-4f2b-b3bd-7becf6d4d4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:29.958637 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431aabd1-4a38-4f2b-b3bd-7becf6d4d4cb admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:29.958914 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431aabd1-4a38-4f2b-b3bd-7becf6d4d4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:29.959186 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431aabd1-4a38-4f2b-b3bd-7becf6d4d4cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:29.959874 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431aabd1-4a38-4f2b-b3bd-7becf6d4d4cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:29.961873 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431aabd1-4a38-4f2b-b3bd-7becf6d4d4cb admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:29.992516 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1414/2816] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:25:29 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:25:29.996540 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815715d-8b93-4138-b0ce-54284054f111 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:30.009173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815715d-8b93-4138-b0ce-54284054f111 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:30.023112 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:30.083558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912f98d-7d9a-4544-98b7-d85ccdcc81a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:30.084198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912f98d-7d9a-4544-98b7-d85ccdcc81a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:30.084502 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912f98d-7d9a-4544-98b7-d85ccdcc81a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:30.084899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912f98d-7d9a-4544-98b7-d85ccdcc81a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:30.085570 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912f98d-7d9a-4544-98b7-d85ccdcc81a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:30.087526 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912f98d-7d9a-4544-98b7-d85ccdcc81a4 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:30.117250 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1403/2817] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:25:29 2022] POST /identity/v3/users => generated 283 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:25:30.118617 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431aabd1-4a38-4f2b-b3bd-7becf6d4d4cb admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:30.131433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431aabd1-4a38-4f2b-b3bd-7becf6d4d4cb admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:30.136593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afb1ae-6e7a-4929-806e-aa9a8ba37be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:30.137426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afb1ae-6e7a-4929-806e-aa9a8ba37be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:30.137901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afb1ae-6e7a-4929-806e-aa9a8ba37be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:30.138234 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1afb1ae-6e7a-4929-806e-aa9a8ba37be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:30.139081 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afb1ae-6e7a-4929-806e-aa9a8ba37be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:30.140976 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1afb1ae-6e7a-4929-806e-aa9a8ba37be5 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:30.153417 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1415/2818] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:25:30.157873 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912f98d-7d9a-4544-98b7-d85ccdcc81a4 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:30.172673 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912f98d-7d9a-4544-98b7-d85ccdcc81a4 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:30.176501 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527917ab-e7a5-4d5b-8909-cd812b47bacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:30.177107 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527917ab-e7a5-4d5b-8909-cd812b47bacd admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:30.177550 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527917ab-e7a5-4d5b-8909-cd812b47bacd admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:30.177864 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527917ab-e7a5-4d5b-8909-cd812b47bacd admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:30.178610 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527917ab-e7a5-4d5b-8909-cd812b47bacd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:30.180317 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527917ab-e7a5-4d5b-8909-cd812b47bacd admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:30.195429 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1404/2819] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:25:30.200379 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afb1ae-6e7a-4929-806e-aa9a8ba37be5 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:30.213473 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1afb1ae-6e7a-4929-806e-aa9a8ba37be5 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:30.218437 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61edb72-d445-42ca-a8c6-4795fde08733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:30.219270 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61edb72-d445-42ca-a8c6-4795fde08733 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:30.219626 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61edb72-d445-42ca-a8c6-4795fde08733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:30.219976 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61edb72-d445-42ca-a8c6-4795fde08733 admin admin] PATH_INFO: `/v3/projects/1d47a0d4007b4fd5b2f2f1e59daf14df/users/2a7a98b21a914310a63a2229e4b5247c/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:30.242765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61edb72-d445-42ca-a8c6-4795fde08733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d47a0d4007b4fd5b2f2f1e59daf14df, user_id=2a7a98b21a914310a63a2229e4b5247c, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:30.245510 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61edb72-d445-42ca-a8c6-4795fde08733 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:30.276651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1416/2820] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:25:30 2022] PUT /identity/v3/projects/1d47a0d4007b4fd5b2f2f1e59daf14df/users/2a7a98b21a914310a63a2229e4b5247c/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:30.288084 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dd0eb-4af1-4481-9527-ca694a503ac6 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:30.288377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dd0eb-4af1-4481-9527-ca694a503ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:30.288699 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dd0eb-4af1-4481-9527-ca694a503ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:30.331503 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-417dd0eb-4af1-4481-9527-ca694a503ac6 None None] MFA Rules not processed for user `2a7a98b21a914310a63a2229e4b5247c`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:30.332793 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-417dd0eb-4af1-4481-9527-ca694a503ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:30.423127 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1405/2821] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:30.430345 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61edb72-d445-42ca-a8c6-4795fde08733 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:30.484171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:30.501350 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61edb72-d445-42ca-a8c6-4795fde08733 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:30.507232 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc3913-fa4d-4e5c-8a31-abec75cdc30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:30.507963 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc3913-fa4d-4e5c-8a31-abec75cdc30e admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:30.508346 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc3913-fa4d-4e5c-8a31-abec75cdc30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:30.508755 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc3913-fa4d-4e5c-8a31-abec75cdc30e admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:30.509531 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc3913-fa4d-4e5c-8a31-abec75cdc30e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:30.511547 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc3913-fa4d-4e5c-8a31-abec75cdc30e admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:30.546123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1417/2822] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:25:30 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:25:30.550090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417dd0eb-4af1-4481-9527-ca694a503ac6 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:30.562935 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417dd0eb-4af1-4481-9527-ca694a503ac6 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:30.633593 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988731b-fab9-482e-8e5e-e494f43358f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:30.637396 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988731b-fab9-482e-8e5e-e494f43358f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:30.638111 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988731b-fab9-482e-8e5e-e494f43358f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:30.638529 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988731b-fab9-482e-8e5e-e494f43358f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:30.639360 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988731b-fab9-482e-8e5e-e494f43358f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:30.643604 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988731b-fab9-482e-8e5e-e494f43358f4 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:30.690479 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1406/2823] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:25:30 2022] POST /identity/v3/users => generated 289 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:25:30.694539 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc3913-fa4d-4e5c-8a31-abec75cdc30e admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:30.706123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc3913-fa4d-4e5c-8a31-abec75cdc30e admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:30.714989 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca2332-45d4-43fc-90b0-3b92af76f5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:30.715726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca2332-45d4-43fc-90b0-3b92af76f5a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:30.716107 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca2332-45d4-43fc-90b0-3b92af76f5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:30.716475 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca2332-45d4-43fc-90b0-3b92af76f5a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:30.717255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca2332-45d4-43fc-90b0-3b92af76f5a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:30.718915 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca2332-45d4-43fc-90b0-3b92af76f5a6 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:30.734392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1418/2824] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:25:30.739679 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988731b-fab9-482e-8e5e-e494f43358f4 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:30.753351 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988731b-fab9-482e-8e5e-e494f43358f4 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:30.760414 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25159218-8525-44ca-9ccd-b24a2e5dbabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:30.761681 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25159218-8525-44ca-9ccd-b24a2e5dbabc admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:30.761961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25159218-8525-44ca-9ccd-b24a2e5dbabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:30.762242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25159218-8525-44ca-9ccd-b24a2e5dbabc admin admin] PATH_INFO: `/v3/projects/ce9187462d5c46469dfc6cb5685a9a2c/users/e69529f8141348ff999faf6ef22baf2a/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:30.778872 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:30.784398 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25159218-8525-44ca-9ccd-b24a2e5dbabc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce9187462d5c46469dfc6cb5685a9a2c, user_id=e69529f8141348ff999faf6ef22baf2a, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:30.788282 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25159218-8525-44ca-9ccd-b24a2e5dbabc admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:30.807174 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1407/2825] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:25:30 2022] PUT /identity/v3/projects/ce9187462d5c46469dfc6cb5685a9a2c/users/e69529f8141348ff999faf6ef22baf2a/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:30.808270 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca2332-45d4-43fc-90b0-3b92af76f5a6 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:30.819324 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca2332-45d4-43fc-90b0-3b92af76f5a6 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:30.897297 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb90737-a77c-4eaf-b115-fa579360997c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:30.898082 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb90737-a77c-4eaf-b115-fa579360997c admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:30.898358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb90737-a77c-4eaf-b115-fa579360997c admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:30.898698 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb90737-a77c-4eaf-b115-fa579360997c admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:30.899466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb90737-a77c-4eaf-b115-fa579360997c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:30.901196 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb90737-a77c-4eaf-b115-fa579360997c admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:30.916727 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1419/2826] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:25:30.918056 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25159218-8525-44ca-9ccd-b24a2e5dbabc admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:30.934077 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25159218-8525-44ca-9ccd-b24a2e5dbabc admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:30.939946 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d19d7e-38aa-4ddd-9029-6fe28e65c33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:30.940646 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d19d7e-38aa-4ddd-9029-6fe28e65c33e admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:30.941074 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d19d7e-38aa-4ddd-9029-6fe28e65c33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:30.941363 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d19d7e-38aa-4ddd-9029-6fe28e65c33e admin admin] PATH_INFO: `/v3/projects/ce9187462d5c46469dfc6cb5685a9a2c/users/e69529f8141348ff999faf6ef22baf2a/roles/35205394c8af4c52a61b12f92ba16644` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:30.943637 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d19d7e-38aa-4ddd-9029-6fe28e65c33e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce9187462d5c46469dfc6cb5685a9a2c, user_id=e69529f8141348ff999faf6ef22baf2a, role_id=35205394c8af4c52a61b12f92ba16644)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:30.946540 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d19d7e-38aa-4ddd-9029-6fe28e65c33e admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:30.958537 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1408/2827] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:25:30 2022] PUT /identity/v3/projects/ce9187462d5c46469dfc6cb5685a9a2c/users/e69529f8141348ff999faf6ef22baf2a/roles/35205394c8af4c52a61b12f92ba16644 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:25:30.969272 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ab379-5b5f-4ef2-b52e-8767b4b9daa7 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:30.969781 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ab379-5b5f-4ef2-b52e-8767b4b9daa7 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:30.970499 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ab379-5b5f-4ef2-b52e-8767b4b9daa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:31.008612 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-1e1ab379-5b5f-4ef2-b52e-8767b4b9daa7 None None] MFA Rules not processed for user `e69529f8141348ff999faf6ef22baf2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:31.009965 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-1e1ab379-5b5f-4ef2-b52e-8767b4b9daa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:31.097621 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1420/2828] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:30 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:31.111845 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d19d7e-38aa-4ddd-9029-6fe28e65c33e admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:31.113646 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-f4d19d7e-38aa-4ddd-9029-6fe28e65c33e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:31.186077 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d19d7e-38aa-4ddd-9029-6fe28e65c33e admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:31.190344 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900c3a9-d324-453a-8faf-4074c480be43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:31.190967 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900c3a9-d324-453a-8faf-4074c480be43 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:31.191246 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900c3a9-d324-453a-8faf-4074c480be43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:31.191516 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900c3a9-d324-453a-8faf-4074c480be43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:31.202584 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900c3a9-d324-453a-8faf-4074c480be43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:31.204106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900c3a9-d324-453a-8faf-4074c480be43 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:31.216426 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1409/2829] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:25:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:25:31.329747 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ab379-5b5f-4ef2-b52e-8767b4b9daa7 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:31.342615 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ab379-5b5f-4ef2-b52e-8767b4b9daa7 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:31.348149 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c171673-6520-42ea-b512-361cfac0e5ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:31.349081 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c171673-6520-42ea-b512-361cfac0e5ef service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:31.349225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c171673-6520-42ea-b512-361cfac0e5ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:31.349694 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c171673-6520-42ea-b512-361cfac0e5ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:31.407633 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:31.424644 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c171673-6520-42ea-b512-361cfac0e5ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:31.426471 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c171673-6520-42ea-b512-361cfac0e5ef service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:31.443427 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1421/2830] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:25:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:25:35.045248 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad481fa2-167e-4c8d-85b9-f86230522871 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:35.045816 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad481fa2-167e-4c8d-85b9-f86230522871 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:35.046325 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad481fa2-167e-4c8d-85b9-f86230522871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:35.072140 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-ad481fa2-167e-4c8d-85b9-f86230522871 None None] MFA Rules not processed for user `e54e713b52e84393b018807f8153922e`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:35.073424 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ad481fa2-167e-4c8d-85b9-f86230522871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:35.146153 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1410/2831] 10.222.0.45 () {58 vars in 967 bytes} [Thu Aug 4 02:25:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:35.159199 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c171673-6520-42ea-b512-361cfac0e5ef service cinder] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:35.160578 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-9c171673-6520-42ea-b512-361cfac0e5ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:35.235174 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c171673-6520-42ea-b512-361cfac0e5ef service cinder] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:35.240655 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95267a1-a81b-4762-8fe5-b571c693e5d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '155a7276efab426cb34175683c475f33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:35.241406 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95267a1-a81b-4762-8fe5-b571c693e5d4 service swift] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:35.242385 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95267a1-a81b-4762-8fe5-b571c693e5d4 service swift] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:35.242385 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95267a1-a81b-4762-8fe5-b571c693e5d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:35.253138 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95267a1-a81b-4762-8fe5-b571c693e5d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:35.254756 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95267a1-a81b-4762-8fe5-b571c693e5d4 service swift] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:35.263634 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1422/2832] 10.222.0.45 () {60 vars in 1384 bytes} [Thu Aug 4 02:25:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 02:25:36.429624 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad481fa2-167e-4c8d-85b9-f86230522871 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:36.493161 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad481fa2-167e-4c8d-85b9-f86230522871 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:36.497666 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a95a60-a86d-4fbe-8c4d-6414808b4805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:36.498296 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a95a60-a86d-4fbe-8c4d-6414808b4805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:36.498574 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a95a60-a86d-4fbe-8c4d-6414808b4805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:36.498846 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a95a60-a86d-4fbe-8c4d-6414808b4805 admin admin] PATH_INFO: `/v3/users/2a7a98b21a914310a63a2229e4b5247c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:36.500011 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a95a60-a86d-4fbe-8c4d-6414808b4805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a7a98b21a914310a63a2229e4b5247c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:36.501961 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a95a60-a86d-4fbe-8c4d-6414808b4805 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:36.529972 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:36.549929 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e2a95a60-a86d-4fbe-8c4d-6414808b4805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a7a98b21a914310a63a2229e4b5247c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:36.550581 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e2a95a60-a86d-4fbe-8c4d-6414808b4805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a7a98b21a914310a63a2229e4b5247c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:36.551190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e2a95a60-a86d-4fbe-8c4d-6414808b4805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a7a98b21a914310a63a2229e4b5247c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:36.563745 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e2a95a60-a86d-4fbe-8c4d-6414808b4805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a7a98b21a914310a63a2229e4b5247c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:36.570641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-e2a95a60-a86d-4fbe-8c4d-6414808b4805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a7a98b21a914310a63a2229e4b5247c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:36.583947 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1411/2833] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:25:36 2022] DELETE /identity/v3/users/2a7a98b21a914310a63a2229e4b5247c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:36.588525 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c38ee6-f97d-42ae-aecc-aaba6212b457 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:36.589175 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c38ee6-f97d-42ae-aecc-aaba6212b457 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:36.589347 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c38ee6-f97d-42ae-aecc-aaba6212b457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:36.607901 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-10c38ee6-f97d-42ae-aecc-aaba6212b457 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:36.609115 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-10c38ee6-f97d-42ae-aecc-aaba6212b457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:36.666895 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1423/2834] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:36.677786 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a95a60-a86d-4fbe-8c4d-6414808b4805 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:36.679190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e2a95a60-a86d-4fbe-8c4d-6414808b4805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:36.744210 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a95a60-a86d-4fbe-8c4d-6414808b4805 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:36.748984 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8a7f5-9f01-4a6b-9b0a-25900858877e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:36.749683 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8a7f5-9f01-4a6b-9b0a-25900858877e service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:36.750855 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8a7f5-9f01-4a6b-9b0a-25900858877e service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:36.751063 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d8a7f5-9f01-4a6b-9b0a-25900858877e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:36.761334 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8a7f5-9f01-4a6b-9b0a-25900858877e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:36.763136 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d8a7f5-9f01-4a6b-9b0a-25900858877e service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:36.774453 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1412/2835] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:25:37.456103 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c38ee6-f97d-42ae-aecc-aaba6212b457 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:37.457643 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-10c38ee6-f97d-42ae-aecc-aaba6212b457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:37.470543 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c38ee6-f97d-42ae-aecc-aaba6212b457 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:37.477447 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:37.478123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:37.478400 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:37.478677 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4 admin admin] PATH_INFO: `/v3/projects/1d47a0d4007b4fd5b2f2f1e59daf14df` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:37.479886 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d47a0d4007b4fd5b2f2f1e59daf14df)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:37.481825 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:37.541267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d47a0d4007b4fd5b2f2f1e59daf14df', 'request_id': 'req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:37.553896 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8a7f5-9f01-4a6b-9b0a-25900858877e service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:37.554630 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d47a0d4007b4fd5b2f2f1e59daf14df', 'request_id': 'req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:37.555672 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-35d8a7f5-9f01-4a6b-9b0a-25900858877e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:37.561728 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d47a0d4007b4fd5b2f2f1e59daf14df', 'request_id': 'req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:37.572818 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:37.574770 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d8a7f5-9f01-4a6b-9b0a-25900858877e service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:37.578726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4 admin admin] The token cache is being invalidate because project 1d47a0d4007b4fd5b2f2f1e59daf14df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:25:37.579028 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:37.580085 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:37.582278 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1424/2836] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:25:37 2022] DELETE /identity/v3/projects/1d47a0d4007b4fd5b2f2f1e59daf14df => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:37.586362 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:37.588384 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:37.637955 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d74eb-424a-4f42-b049-2d16ad087ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:37.638995 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d74eb-424a-4f42-b049-2d16ad087ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:37.639280 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d74eb-424a-4f42-b049-2d16ad087ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:37.639557 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09d74eb-424a-4f42-b049-2d16ad087ba6 admin admin] PATH_INFO: `/v3/users/b18a6ae879c14eabae7a7a6db120831a` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:37.641459 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d74eb-424a-4f42-b049-2d16ad087ba6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b18a6ae879c14eabae7a7a6db120831a)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:37.643407 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09d74eb-424a-4f42-b049-2d16ad087ba6 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:37.660845 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7da8a7-b8ba-4e07-805d-c90788b5a0e4 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:37.665519 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e223a8-c503-4441-92da-fc53b786322c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:37.666157 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e223a8-c503-4441-92da-fc53b786322c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:37.666433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e223a8-c503-4441-92da-fc53b786322c admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:37.666710 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e223a8-c503-4441-92da-fc53b786322c admin admin] PATH_INFO: `/v3/users/e69529f8141348ff999faf6ef22baf2a` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:37.668033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e223a8-c503-4441-92da-fc53b786322c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e69529f8141348ff999faf6ef22baf2a)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:37.670025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e223a8-c503-4441-92da-fc53b786322c admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:37.698737 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c09d74eb-424a-4f42-b049-2d16ad087ba6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b18a6ae879c14eabae7a7a6db120831a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:37.699821 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c09d74eb-424a-4f42-b049-2d16ad087ba6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b18a6ae879c14eabae7a7a6db120831a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:37.701377 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c09d74eb-424a-4f42-b049-2d16ad087ba6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b18a6ae879c14eabae7a7a6db120831a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:37.705091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:37.719987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c09d74eb-424a-4f42-b049-2d16ad087ba6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b18a6ae879c14eabae7a7a6db120831a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:37.727569 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-c09d74eb-424a-4f42-b049-2d16ad087ba6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b18a6ae879c14eabae7a7a6db120831a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:37.730033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-78e223a8-c503-4441-92da-fc53b786322c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e69529f8141348ff999faf6ef22baf2a'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:37.738726 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-78e223a8-c503-4441-92da-fc53b786322c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e69529f8141348ff999faf6ef22baf2a'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:37.739674 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-78e223a8-c503-4441-92da-fc53b786322c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e69529f8141348ff999faf6ef22baf2a'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:37.741115 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1413/2837] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:25:37 2022] DELETE /identity/v3/users/b18a6ae879c14eabae7a7a6db120831a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:37.754939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-78e223a8-c503-4441-92da-fc53b786322c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e69529f8141348ff999faf6ef22baf2a'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:37.756301 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-78e223a8-c503-4441-92da-fc53b786322c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e69529f8141348ff999faf6ef22baf2a'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:37.771169 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1425/2838] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:25:37 2022] DELETE /identity/v3/users/e69529f8141348ff999faf6ef22baf2a => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:25:37.939757 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d74eb-424a-4f42-b049-2d16ad087ba6 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:37.941392 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-c09d74eb-424a-4f42-b049-2d16ad087ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:38.019073 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09d74eb-424a-4f42-b049-2d16ad087ba6 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:38.093185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ce2fa-7dc0-4aed-a756-31ca1999b6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:38.097793 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ce2fa-7dc0-4aed-a756-31ca1999b6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:38.097793 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ce2fa-7dc0-4aed-a756-31ca1999b6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:38.097793 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ce2fa-7dc0-4aed-a756-31ca1999b6ce admin admin] PATH_INFO: `/v3/projects/f7d4919e934044b7a97dce3ad1cac880` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:38.098811 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e223a8-c503-4441-92da-fc53b786322c admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:38.100770 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-78e223a8-c503-4441-92da-fc53b786322c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:38.106855 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ce2fa-7dc0-4aed-a756-31ca1999b6ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7d4919e934044b7a97dce3ad1cac880)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:38.109123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ce2fa-7dc0-4aed-a756-31ca1999b6ce admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:38.120182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e223a8-c503-4441-92da-fc53b786322c admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:38.125894 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59fbaa-a86f-4c4c-9700-a09af331df7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:38.127429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59fbaa-a86f-4c4c-9700-a09af331df7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:38.128066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59fbaa-a86f-4c4c-9700-a09af331df7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:38.128514 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59fbaa-a86f-4c4c-9700-a09af331df7a admin admin] PATH_INFO: `/v3/projects/ce9187462d5c46469dfc6cb5685a9a2c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:38.131547 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59fbaa-a86f-4c4c-9700-a09af331df7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce9187462d5c46469dfc6cb5685a9a2c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:38.135045 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f59fbaa-a86f-4c4c-9700-a09af331df7a admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:38.177191 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-619ce2fa-7dc0-4aed-a756-31ca1999b6ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7d4919e934044b7a97dce3ad1cac880', 'request_id': 'req-619ce2fa-7dc0-4aed-a756-31ca1999b6ce'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:38.186568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-619ce2fa-7dc0-4aed-a756-31ca1999b6ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7d4919e934044b7a97dce3ad1cac880', 'request_id': 'req-619ce2fa-7dc0-4aed-a756-31ca1999b6ce'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:38.193598 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-619ce2fa-7dc0-4aed-a756-31ca1999b6ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7d4919e934044b7a97dce3ad1cac880', 'request_id': 'req-619ce2fa-7dc0-4aed-a756-31ca1999b6ce'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:38.196426 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3f59fbaa-a86f-4c4c-9700-a09af331df7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce9187462d5c46469dfc6cb5685a9a2c', 'request_id': 'req-3f59fbaa-a86f-4c4c-9700-a09af331df7a'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:38.207340 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-619ce2fa-7dc0-4aed-a756-31ca1999b6ce admin admin] The token cache is being invalidate because project f7d4919e934044b7a97dce3ad1cac880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:25:38.207705 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-619ce2fa-7dc0-4aed-a756-31ca1999b6ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:38.208255 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3f59fbaa-a86f-4c4c-9700-a09af331df7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce9187462d5c46469dfc6cb5685a9a2c', 'request_id': 'req-3f59fbaa-a86f-4c4c-9700-a09af331df7a'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:38.208494 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-619ce2fa-7dc0-4aed-a756-31ca1999b6ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:38.210837 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1414/2839] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:25:37 2022] DELETE /identity/v3/projects/f7d4919e934044b7a97dce3ad1cac880 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:38.216402 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3f59fbaa-a86f-4c4c-9700-a09af331df7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce9187462d5c46469dfc6cb5685a9a2c', 'request_id': 'req-3f59fbaa-a86f-4c4c-9700-a09af331df7a'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:38.227769 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:38.232810 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3f59fbaa-a86f-4c4c-9700-a09af331df7a admin admin] The token cache is being invalidate because project ce9187462d5c46469dfc6cb5685a9a2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:25:38.233859 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3f59fbaa-a86f-4c4c-9700-a09af331df7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:38.235520 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-3f59fbaa-a86f-4c4c-9700-a09af331df7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:25:38.237827 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1426/2840] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:25:38 2022] DELETE /identity/v3/projects/ce9187462d5c46469dfc6cb5685a9a2c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:25:38.246289 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf647d0e-4ab5-429f-af77-cb67b6949fdf None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:38.246568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf647d0e-4ab5-429f-af77-cb67b6949fdf None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:38.246884 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf647d0e-4ab5-429f-af77-cb67b6949fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:38.263221 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:38.271190 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-bf647d0e-4ab5-429f-af77-cb67b6949fdf None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:38.272464 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-bf647d0e-4ab5-429f-af77-cb67b6949fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:38.355044 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1415/2841] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:38.363304 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085eaa5f-16be-4e6e-8c73-7dcbdecbdf7a None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:38.363688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085eaa5f-16be-4e6e-8c73-7dcbdecbdf7a None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:38.364040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085eaa5f-16be-4e6e-8c73-7dcbdecbdf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:38.386627 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-085eaa5f-16be-4e6e-8c73-7dcbdecbdf7a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:38.388151 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-085eaa5f-16be-4e6e-8c73-7dcbdecbdf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:38.400225 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1427/2842] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:38.405987 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf647d0e-4ab5-429f-af77-cb67b6949fdf None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:38.416641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf647d0e-4ab5-429f-af77-cb67b6949fdf None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:38.436895 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2c70d-bd58-4cf6-91a1-fd6ff08d29c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:38.438155 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2c70d-bd58-4cf6-91a1-fd6ff08d29c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:38.438504 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2c70d-bd58-4cf6-91a1-fd6ff08d29c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:38.438911 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2c70d-bd58-4cf6-91a1-fd6ff08d29c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:38.440334 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2c70d-bd58-4cf6-91a1-fd6ff08d29c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80673) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:25:38.440863 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2c70d-bd58-4cf6-91a1-fd6ff08d29c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:38.444133 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2c70d-bd58-4cf6-91a1-fd6ff08d29c6 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:38.460080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1416/2843] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:25:38.464566 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085eaa5f-16be-4e6e-8c73-7dcbdecbdf7a None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:38.474466 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085eaa5f-16be-4e6e-8c73-7dcbdecbdf7a None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:38.479178 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477eb4b-b29d-49f8-b53f-d0703f99d5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:38.479782 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477eb4b-b29d-49f8-b53f-d0703f99d5f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:38.480084 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477eb4b-b29d-49f8-b53f-d0703f99d5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:38.480360 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477eb4b-b29d-49f8-b53f-d0703f99d5f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:38.481246 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477eb4b-b29d-49f8-b53f-d0703f99d5f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:38.483164 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2477eb4b-b29d-49f8-b53f-d0703f99d5f3 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:38.511593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1428/2844] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:25:38 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 04 02:25:38.516545 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2c70d-bd58-4cf6-91a1-fd6ff08d29c6 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:38.526856 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2c70d-bd58-4cf6-91a1-fd6ff08d29c6 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:38.583406 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4763039-6a88-44b6-a686-f9200f9a467d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:38.584093 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4763039-6a88-44b6-a686-f9200f9a467d admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:38.584290 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4763039-6a88-44b6-a686-f9200f9a467d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:38.584566 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4763039-6a88-44b6-a686-f9200f9a467d admin admin] PATH_INFO: `/v3/users` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:38.585266 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4763039-6a88-44b6-a686-f9200f9a467d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:38.587242 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4763039-6a88-44b6-a686-f9200f9a467d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:38.608198 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1417/2845] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:25:38 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:25:38.612280 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477eb4b-b29d-49f8-b53f-d0703f99d5f3 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:38.622411 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477eb4b-b29d-49f8-b53f-d0703f99d5f3 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:38.627433 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5400d80-d017-4506-8c07-41933358363f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:38.628039 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5400d80-d017-4506-8c07-41933358363f admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:38.628322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5400d80-d017-4506-8c07-41933358363f admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:38.628601 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5400d80-d017-4506-8c07-41933358363f admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:38.629319 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5400d80-d017-4506-8c07-41933358363f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:38.630903 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5400d80-d017-4506-8c07-41933358363f admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:38.641277 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1429/2846] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:25:38.645306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4763039-6a88-44b6-a686-f9200f9a467d admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:38.655437 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4763039-6a88-44b6-a686-f9200f9a467d admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:38.660263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a8b94-5a86-4c14-ae37-5a392ef0341d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:38.660900 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a8b94-5a86-4c14-ae37-5a392ef0341d admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:38.661185 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a8b94-5a86-4c14-ae37-5a392ef0341d admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:38.661457 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a8b94-5a86-4c14-ae37-5a392ef0341d admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:38.662181 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a8b94-5a86-4c14-ae37-5a392ef0341d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:38.663773 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a8b94-5a86-4c14-ae37-5a392ef0341d admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:38.674754 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1418/2847] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 04 02:25:38.679244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5400d80-d017-4506-8c07-41933358363f admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:38.690336 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5400d80-d017-4506-8c07-41933358363f admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:38.695244 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8aef1-fa63-4a4f-ae48-d0ed02e39be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:38.696004 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8aef1-fa63-4a4f-ae48-d0ed02e39be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:38.696289 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8aef1-fa63-4a4f-ae48-d0ed02e39be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:38.696573 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a8aef1-fa63-4a4f-ae48-d0ed02e39be1 admin admin] PATH_INFO: `/v3/projects/b8fd26658f57474aba12bceb6b9c7176/users/ccc39daa4b074ddeb97755f1b1535f50/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:38.712012 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8aef1-fa63-4a4f-ae48-d0ed02e39be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8fd26658f57474aba12bceb6b9c7176, user_id=ccc39daa4b074ddeb97755f1b1535f50, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:38.714694 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a8aef1-fa63-4a4f-ae48-d0ed02e39be1 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:38.728137 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1430/2848] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:25:38 2022] PUT /identity/v3/projects/b8fd26658f57474aba12bceb6b9c7176/users/ccc39daa4b074ddeb97755f1b1535f50/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:25:38.741607 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e95aef-0c5a-465b-8dd4-8b4d1df253bc None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:38.741935 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e95aef-0c5a-465b-8dd4-8b4d1df253bc None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:38.742428 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e95aef-0c5a-465b-8dd4-8b4d1df253bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:38.764988 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-a0e95aef-0c5a-465b-8dd4-8b4d1df253bc None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:38.766224 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a0e95aef-0c5a-465b-8dd4-8b4d1df253bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:38.829133 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1419/2849] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 04 02:25:38.838699 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8aef1-fa63-4a4f-ae48-d0ed02e39be1 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:38.840407 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-17a8aef1-fa63-4a4f-ae48-d0ed02e39be1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:38.897129 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a8aef1-fa63-4a4f-ae48-d0ed02e39be1 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:38.900758 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e9260-b534-4197-ac20-6386cae67921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:38.901365 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e9260-b534-4197-ac20-6386cae67921 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:38.901645 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e9260-b534-4197-ac20-6386cae67921 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:38.901935 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e9260-b534-4197-ac20-6386cae67921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:38.910926 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e9260-b534-4197-ac20-6386cae67921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:38.912340 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e9260-b534-4197-ac20-6386cae67921 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:38.922822 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1431/2850] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:25:39.584505 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232dd97-a3d8-4b12-8223-7630632b8c71 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:39.585090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232dd97-a3d8-4b12-8223-7630632b8c71 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:39.585090 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232dd97-a3d8-4b12-8223-7630632b8c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:39.607346 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:39.616324 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-e232dd97-a3d8-4b12-8223-7630632b8c71 None None] MFA Rules not processed for user `ccc39daa4b074ddeb97755f1b1535f50`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:39.618026 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e232dd97-a3d8-4b12-8223-7630632b8c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:39.685465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1420/2851] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:39.696392 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e9260-b534-4197-ac20-6386cae67921 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:25:39.697824 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-4c0e9260-b534-4197-ac20-6386cae67921 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:39.757267 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e9260-b534-4197-ac20-6386cae67921 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:25:39.761331 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eda036-e513-4d4f-a247-c7b40cc11360 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:25:39.762026 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eda036-e513-4d4f-a247-c7b40cc11360 service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:39.762435 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eda036-e513-4d4f-a247-c7b40cc11360 service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:39.762724 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eda036-e513-4d4f-a247-c7b40cc11360 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:39.772332 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eda036-e513-4d4f-a247-c7b40cc11360 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:25:39.773878 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eda036-e513-4d4f-a247-c7b40cc11360 service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:25:39.783337 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1432/2852] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:25:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:25:41.143106 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966daf6f-9e1f-46da-b822-26d1c00d5291 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:25:41.143610 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966daf6f-9e1f-46da-b822-26d1c00d5291 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:25:41.143709 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966daf6f-9e1f-46da-b822-26d1c00d5291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:25:41.161914 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-966daf6f-9e1f-46da-b822-26d1c00d5291 None None] MFA Rules not processed for user `ccc39daa4b074ddeb97755f1b1535f50`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:25:41.163108 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-966daf6f-9e1f-46da-b822-26d1c00d5291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:25:41.169818 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1421/2853] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:25:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:25:49.778839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=79,keystone:INSERT=2 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:25:51.156767 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=30 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:26:41.917784 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44fd72-634f-4682-822a-dc313bfa5ab2 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:26:41.918338 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44fd72-634f-4682-822a-dc313bfa5ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:26:41.918338 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44fd72-634f-4682-822a-dc313bfa5ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:26:41.937040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-cd44fd72-634f-4682-822a-dc313bfa5ab2 None None] MFA Rules not processed for user `ccc39daa4b074ddeb97755f1b1535f50`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:26:41.938246 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-cd44fd72-634f-4682-822a-dc313bfa5ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:26:41.945446 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1433/2854] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:26:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:26:51.931727 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=3 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:26:54.326041 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966daf6f-9e1f-46da-b822-26d1c00d5291 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:26:54.337270 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966daf6f-9e1f-46da-b822-26d1c00d5291 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:26:54.341372 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ad8b8-63cc-4e0e-bde3-e8319cee983e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:26:54.342585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ad8b8-63cc-4e0e-bde3-e8319cee983e service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:26:54.342585 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ad8b8-63cc-4e0e-bde3-e8319cee983e service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:26:54.342796 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ad8b8-63cc-4e0e-bde3-e8319cee983e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:26:54.346849 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a70ad8b8-63cc-4e0e-bde3-e8319cee983e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:26:54.442599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ad8b8-63cc-4e0e-bde3-e8319cee983e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:26:54.444205 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ad8b8-63cc-4e0e-bde3-e8319cee983e service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:26:54.474839 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1422/2855] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:26:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:27:03.383710 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44fd72-634f-4682-822a-dc313bfa5ab2 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:03.394165 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44fd72-634f-4682-822a-dc313bfa5ab2 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:03.397746 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e613852-1c73-474c-8a8d-6f397a77f46d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:03.398364 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e613852-1c73-474c-8a8d-6f397a77f46d service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:03.398637 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e613852-1c73-474c-8a8d-6f397a77f46d service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:03.398907 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e613852-1c73-474c-8a8d-6f397a77f46d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:03.400640 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-6e613852-1c73-474c-8a8d-6f397a77f46d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:27:03.414956 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e613852-1c73-474c-8a8d-6f397a77f46d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:03.416387 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e613852-1c73-474c-8a8d-6f397a77f46d service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:03.427585 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1434/2856] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:27:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:27:04.467981 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=43 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:27:07.236029 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75920a-6859-44ea-813a-c18daaf1d493 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:07.236596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75920a-6859-44ea-813a-c18daaf1d493 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:07.236596 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d75920a-6859-44ea-813a-c18daaf1d493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:07.255430 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-7d75920a-6859-44ea-813a-c18daaf1d493 None None] MFA Rules not processed for user `683f8f120df946f59505455a9ab2dc2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:27:07.256610 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7d75920a-6859-44ea-813a-c18daaf1d493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:27:07.309911 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1423/2857] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:27:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:27:11.518092 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab8eb5-a6c2-4735-af8f-3393c0476a0e None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:11.519070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab8eb5-a6c2-4735-af8f-3393c0476a0e None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:11.519602 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ab8eb5-a6c2-4735-af8f-3393c0476a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:11.581218 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-c4ab8eb5-a6c2-4735-af8f-3393c0476a0e None None] MFA Rules not processed for user `e54e713b52e84393b018807f8153922e`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:27:11.583593 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c4ab8eb5-a6c2-4735-af8f-3393c0476a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:27:11.673678 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1435/2858] 10.222.0.45 () {58 vars in 967 bytes} [Thu Aug 4 02:27:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:27:11.752579 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75920a-6859-44ea-813a-c18daaf1d493 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:11.765269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-7d75920a-6859-44ea-813a-c18daaf1d493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:27:11.859070 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d75920a-6859-44ea-813a-c18daaf1d493 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:11.864103 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dbb46-a3fa-4912-826a-cb82811ac363 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '155a7276efab426cb34175683c475f33', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:11.864748 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dbb46-a3fa-4912-826a-cb82811ac363 service swift] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:11.865032 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dbb46-a3fa-4912-826a-cb82811ac363 service swift] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:11.865307 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948dbb46-a3fa-4912-826a-cb82811ac363 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:11.875275 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dbb46-a3fa-4912-826a-cb82811ac363 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:11.876986 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948dbb46-a3fa-4912-826a-cb82811ac363 service swift] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:11.885466 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1424/2859] 10.222.0.45 () {60 vars in 1384 bytes} [Thu Aug 4 02:27:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 04 02:27:19.763647 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab8eb5-a6c2-4735-af8f-3393c0476a0e None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:19.765123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-c4ab8eb5-a6c2-4735-af8f-3393c0476a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:27:19.779785 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ab8eb5-a6c2-4735-af8f-3393c0476a0e None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:19.784961 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7dbadf-a231-4665-a466-12f04f1f7679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:19.785599 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dbadf-a231-4665-a466-12f04f1f7679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:19.785885 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dbadf-a231-4665-a466-12f04f1f7679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:19.786166 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dbadf-a231-4665-a466-12f04f1f7679 admin admin] PATH_INFO: `/v3/users/fb3a977fea2c499eb915bc059ab01a79` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:19.787346 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7dbadf-a231-4665-a466-12f04f1f7679 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb3a977fea2c499eb915bc059ab01a79)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:19.789312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7dbadf-a231-4665-a466-12f04f1f7679 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:19.835205 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-ed7dbadf-a231-4665-a466-12f04f1f7679 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb3a977fea2c499eb915bc059ab01a79'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:19.843109 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-ed7dbadf-a231-4665-a466-12f04f1f7679 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb3a977fea2c499eb915bc059ab01a79'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:19.844329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-ed7dbadf-a231-4665-a466-12f04f1f7679 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb3a977fea2c499eb915bc059ab01a79'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:19.856942 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-ed7dbadf-a231-4665-a466-12f04f1f7679 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb3a977fea2c499eb915bc059ab01a79'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:19.857712 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-ed7dbadf-a231-4665-a466-12f04f1f7679 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb3a977fea2c499eb915bc059ab01a79'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:19.871435 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1436/2860] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:27:19 2022] DELETE /identity/v3/users/fb3a977fea2c499eb915bc059ab01a79 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:27:20.044884 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dbb46-a3fa-4912-826a-cb82811ac363 service swift] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:20.057295 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948dbb46-a3fa-4912-826a-cb82811ac363 service swift] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:20.117065 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6aac50-2a23-460f-8d24-4c366dfc054f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:20.117698 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6aac50-2a23-460f-8d24-4c366dfc054f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:20.117980 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6aac50-2a23-460f-8d24-4c366dfc054f admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:20.118249 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6aac50-2a23-460f-8d24-4c366dfc054f admin admin] PATH_INFO: `/v3/projects/06f7c91a924c447fbca1b6a784d7b167` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:20.119409 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6aac50-2a23-460f-8d24-4c366dfc054f admin admin] RBAC: Authorizing `identity:delete_project(project_id=06f7c91a924c447fbca1b6a784d7b167)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:20.121359 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6aac50-2a23-460f-8d24-4c366dfc054f admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:20.158586 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:27:20.168028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-2a6aac50-2a23-460f-8d24-4c366dfc054f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06f7c91a924c447fbca1b6a784d7b167', 'request_id': 'req-2a6aac50-2a23-460f-8d24-4c366dfc054f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:20.177603 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-2a6aac50-2a23-460f-8d24-4c366dfc054f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06f7c91a924c447fbca1b6a784d7b167', 'request_id': 'req-2a6aac50-2a23-460f-8d24-4c366dfc054f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:20.183599 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-2a6aac50-2a23-460f-8d24-4c366dfc054f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06f7c91a924c447fbca1b6a784d7b167', 'request_id': 'req-2a6aac50-2a23-460f-8d24-4c366dfc054f'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:20.195393 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-2a6aac50-2a23-460f-8d24-4c366dfc054f admin admin] The token cache is being invalidate because project 06f7c91a924c447fbca1b6a784d7b167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:27:20.195685 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-2a6aac50-2a23-460f-8d24-4c366dfc054f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:20.196327 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-2a6aac50-2a23-460f-8d24-4c366dfc054f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:20.197854 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1425/2861] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:27:20 2022] DELETE /identity/v3/projects/06f7c91a924c447fbca1b6a784d7b167 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:27:20.201927 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7dbadf-a231-4665-a466-12f04f1f7679 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:20.203513 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-ed7dbadf-a231-4665-a466-12f04f1f7679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:27:20.219070 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7dbadf-a231-4665-a466-12f04f1f7679 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:20.265625 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:27:20.282145 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea39eba-0da2-4539-9af2-555b1dc99759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:20.282909 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea39eba-0da2-4539-9af2-555b1dc99759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:20.283259 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea39eba-0da2-4539-9af2-555b1dc99759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:20.283634 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea39eba-0da2-4539-9af2-555b1dc99759 admin admin] PATH_INFO: `/v3/users/46053624901c4b55acb1089e9dbbb5ff` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:20.284939 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea39eba-0da2-4539-9af2-555b1dc99759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46053624901c4b55acb1089e9dbbb5ff)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:20.286964 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea39eba-0da2-4539-9af2-555b1dc99759 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:20.339648 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fea39eba-0da2-4539-9af2-555b1dc99759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46053624901c4b55acb1089e9dbbb5ff'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:20.347603 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fea39eba-0da2-4539-9af2-555b1dc99759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46053624901c4b55acb1089e9dbbb5ff'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:20.348327 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fea39eba-0da2-4539-9af2-555b1dc99759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46053624901c4b55acb1089e9dbbb5ff'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:20.361516 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fea39eba-0da2-4539-9af2-555b1dc99759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46053624901c4b55acb1089e9dbbb5ff'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:20.362283 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-fea39eba-0da2-4539-9af2-555b1dc99759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46053624901c4b55acb1089e9dbbb5ff'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:20.383224 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1437/2862] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:27:20 2022] DELETE /identity/v3/users/46053624901c4b55acb1089e9dbbb5ff => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:27:20.559738 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6aac50-2a23-460f-8d24-4c366dfc054f admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:20.561292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-2a6aac50-2a23-460f-8d24-4c366dfc054f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:27:20.576838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6aac50-2a23-460f-8d24-4c366dfc054f admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:20.637202 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52106990-43a3-4604-a7fe-1d82ff2b158c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:20.638062 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52106990-43a3-4604-a7fe-1d82ff2b158c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:20.638173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52106990-43a3-4604-a7fe-1d82ff2b158c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:20.638401 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52106990-43a3-4604-a7fe-1d82ff2b158c admin admin] PATH_INFO: `/v3/projects/72c769241feb4f20a536bf1cb3b8a13b` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:20.639613 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52106990-43a3-4604-a7fe-1d82ff2b158c admin admin] RBAC: Authorizing `identity:delete_project(project_id=72c769241feb4f20a536bf1cb3b8a13b)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:20.641689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52106990-43a3-4604-a7fe-1d82ff2b158c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:20.691320 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-52106990-43a3-4604-a7fe-1d82ff2b158c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72c769241feb4f20a536bf1cb3b8a13b', 'request_id': 'req-52106990-43a3-4604-a7fe-1d82ff2b158c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:20.700173 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-52106990-43a3-4604-a7fe-1d82ff2b158c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72c769241feb4f20a536bf1cb3b8a13b', 'request_id': 'req-52106990-43a3-4604-a7fe-1d82ff2b158c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:20.706402 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-52106990-43a3-4604-a7fe-1d82ff2b158c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72c769241feb4f20a536bf1cb3b8a13b', 'request_id': 'req-52106990-43a3-4604-a7fe-1d82ff2b158c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:20.719403 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-52106990-43a3-4604-a7fe-1d82ff2b158c admin admin] The token cache is being invalidate because project 72c769241feb4f20a536bf1cb3b8a13b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:27:20.719686 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-52106990-43a3-4604-a7fe-1d82ff2b158c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:20.720946 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-52106990-43a3-4604-a7fe-1d82ff2b158c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:20.722454 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1426/2863] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:27:20 2022] DELETE /identity/v3/projects/72c769241feb4f20a536bf1cb3b8a13b => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:27:30.379765 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=32,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:27:30.714853 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:27:37.847516 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea39eba-0da2-4539-9af2-555b1dc99759 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:37.849018 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-fea39eba-0da2-4539-9af2-555b1dc99759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:27:37.862896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea39eba-0da2-4539-9af2-555b1dc99759 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:37.919100 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61b3303-031e-4dd1-865c-9273e0115da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:37.920328 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3303-031e-4dd1-865c-9273e0115da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:37.920633 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3303-031e-4dd1-865c-9273e0115da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:37.921029 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61b3303-031e-4dd1-865c-9273e0115da8 admin admin] PATH_INFO: `/v3/users/ccc39daa4b074ddeb97755f1b1535f50` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:37.922956 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b3303-031e-4dd1-865c-9273e0115da8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccc39daa4b074ddeb97755f1b1535f50)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:37.924911 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61b3303-031e-4dd1-865c-9273e0115da8 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:37.971706 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-c61b3303-031e-4dd1-865c-9273e0115da8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccc39daa4b074ddeb97755f1b1535f50'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:37.978474 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-c61b3303-031e-4dd1-865c-9273e0115da8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccc39daa4b074ddeb97755f1b1535f50'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:37.979138 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-c61b3303-031e-4dd1-865c-9273e0115da8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccc39daa4b074ddeb97755f1b1535f50'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:37.991222 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-c61b3303-031e-4dd1-865c-9273e0115da8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccc39daa4b074ddeb97755f1b1535f50'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:37.991863 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-c61b3303-031e-4dd1-865c-9273e0115da8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccc39daa4b074ddeb97755f1b1535f50'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:38.005146 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1438/2864] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:27:37 2022] DELETE /identity/v3/users/ccc39daa4b074ddeb97755f1b1535f50 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:27:38.185912 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52106990-43a3-4604-a7fe-1d82ff2b158c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:38.187507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-52106990-43a3-4604-a7fe-1d82ff2b158c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:27:38.200943 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52106990-43a3-4604-a7fe-1d82ff2b158c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:38.260654 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce54bc25-4529-4e29-93c0-5899f2de372c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:38.261292 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54bc25-4529-4e29-93c0-5899f2de372c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:38.261568 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54bc25-4529-4e29-93c0-5899f2de372c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:38.261838 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce54bc25-4529-4e29-93c0-5899f2de372c admin admin] PATH_INFO: `/v3/projects/b8fd26658f57474aba12bceb6b9c7176` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:38.262973 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54bc25-4529-4e29-93c0-5899f2de372c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8fd26658f57474aba12bceb6b9c7176)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:38.264893 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce54bc25-4529-4e29-93c0-5899f2de372c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:38.312024 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ce54bc25-4529-4e29-93c0-5899f2de372c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8fd26658f57474aba12bceb6b9c7176', 'request_id': 'req-ce54bc25-4529-4e29-93c0-5899f2de372c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:38.319389 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ce54bc25-4529-4e29-93c0-5899f2de372c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8fd26658f57474aba12bceb6b9c7176', 'request_id': 'req-ce54bc25-4529-4e29-93c0-5899f2de372c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:38.325306 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ce54bc25-4529-4e29-93c0-5899f2de372c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8fd26658f57474aba12bceb6b9c7176', 'request_id': 'req-ce54bc25-4529-4e29-93c0-5899f2de372c'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:38.336157 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ce54bc25-4529-4e29-93c0-5899f2de372c admin admin] The token cache is being invalidate because project b8fd26658f57474aba12bceb6b9c7176 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:27:38.336458 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ce54bc25-4529-4e29-93c0-5899f2de372c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:38.337105 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-ce54bc25-4529-4e29-93c0-5899f2de372c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:38.338615 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1427/2865] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:27:38 2022] DELETE /identity/v3/projects/b8fd26658f57474aba12bceb6b9c7176 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:27:38.343973 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb894d-e55d-4370-9115-82b357334b6c None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:38.344795 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb894d-e55d-4370-9115-82b357334b6c None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:38.345074 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb894d-e55d-4370-9115-82b357334b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:38.366279 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-bbfb894d-e55d-4370-9115-82b357334b6c None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:27:38.367501 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-bbfb894d-e55d-4370-9115-82b357334b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:27:38.425002 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1439/2866] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:27:38.433782 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a688c12-4e62-4b65-ba5c-e8170f893c3a None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:38.434077 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a688c12-4e62-4b65-ba5c-e8170f893c3a None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:38.434356 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a688c12-4e62-4b65-ba5c-e8170f893c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:38.454218 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-5a688c12-4e62-4b65-ba5c-e8170f893c3a None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:27:38.455424 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-5a688c12-4e62-4b65-ba5c-e8170f893c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:27:38.463290 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1428/2867] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:27:38.467651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb894d-e55d-4370-9115-82b357334b6c None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:38.478256 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb894d-e55d-4370-9115-82b357334b6c None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:38.482836 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac2320-a8d7-40ed-abc3-5e35e221a249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:38.483432 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac2320-a8d7-40ed-abc3-5e35e221a249 admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:38.483715 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac2320-a8d7-40ed-abc3-5e35e221a249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:38.483993 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac2320-a8d7-40ed-abc3-5e35e221a249 admin admin] PATH_INFO: `/v3/domains` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:38.484727 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac2320-a8d7-40ed-abc3-5e35e221a249 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80672) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 04 02:27:38.485588 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac2320-a8d7-40ed-abc3-5e35e221a249 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:38.487170 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac2320-a8d7-40ed-abc3-5e35e221a249 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:38.498740 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1440/2868] 10.222.0.45 () {58 vars in 1153 bytes} [Thu Aug 4 02:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 04 02:27:38.503082 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a688c12-4e62-4b65-ba5c-e8170f893c3a None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:38.512249 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a688c12-4e62-4b65-ba5c-e8170f893c3a None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:38.517039 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbea08-573f-4b0e-844d-841b9f6e04e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:38.517641 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbea08-573f-4b0e-844d-841b9f6e04e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:38.517917 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbea08-573f-4b0e-844d-841b9f6e04e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:38.518192 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbea08-573f-4b0e-844d-841b9f6e04e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:38.518845 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbea08-573f-4b0e-844d-841b9f6e04e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:38.520754 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbea08-573f-4b0e-844d-841b9f6e04e7 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:38.558178 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1429/2869] 10.222.0.45 () {60 vars in 1153 bytes} [Thu Aug 4 02:27:38 2022] POST /identity/v3/projects => generated 397 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:27:38.562458 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac2320-a8d7-40ed-abc3-5e35e221a249 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:38.572517 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac2320-a8d7-40ed-abc3-5e35e221a249 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:38.576077 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:27:38.631896 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea1ac7-4ac6-4945-a36e-898a0788c4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:38.632516 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea1ac7-4ac6-4945-a36e-898a0788c4ea admin admin] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:38.632839 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea1ac7-4ac6-4945-a36e-898a0788c4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:38.633135 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea1ac7-4ac6-4945-a36e-898a0788c4ea admin admin] PATH_INFO: `/v3/users` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:38.633817 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea1ac7-4ac6-4945-a36e-898a0788c4ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:38.635716 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea1ac7-4ac6-4945-a36e-898a0788c4ea admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:38.661526 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1441/2870] 10.222.0.45 () {60 vars in 1144 bytes} [Thu Aug 4 02:27:38 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 04 02:27:38.663483 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbea08-573f-4b0e-844d-841b9f6e04e7 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:38.673572 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbea08-573f-4b0e-844d-841b9f6e04e7 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:38.678195 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043b0bd-3b97-43c8-8d76-852baa4ffce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:38.678732 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043b0bd-3b97-43c8-8d76-852baa4ffce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:38.679005 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043b0bd-3b97-43c8-8d76-852baa4ffce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:38.679270 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043b0bd-3b97-43c8-8d76-852baa4ffce3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:38.679981 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043b0bd-3b97-43c8-8d76-852baa4ffce3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:38.681576 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043b0bd-3b97-43c8-8d76-852baa4ffce3 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:38.691215 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1430/2871] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:27:38.695128 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea1ac7-4ac6-4945-a36e-898a0788c4ea admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:38.706061 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea1ac7-4ac6-4945-a36e-898a0788c4ea admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:38.710682 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489bc0b-69f2-4b58-a1c0-336c2ee86e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:38.711282 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489bc0b-69f2-4b58-a1c0-336c2ee86e3b admin admin] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:38.711563 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489bc0b-69f2-4b58-a1c0-336c2ee86e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:38.711833 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489bc0b-69f2-4b58-a1c0-336c2ee86e3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:38.712518 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489bc0b-69f2-4b58-a1c0-336c2ee86e3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:38.714128 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489bc0b-69f2-4b58-a1c0-336c2ee86e3b admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:38.725468 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1442/2872] 10.222.0.45 () {58 vars in 1122 bytes} [Thu Aug 4 02:27:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 04 02:27:38.729820 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043b0bd-3b97-43c8-8d76-852baa4ffce3 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:38.740206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043b0bd-3b97-43c8-8d76-852baa4ffce3 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:38.744852 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27301329-3ec4-45dc-9a5c-4ce74023751c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:38.745507 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27301329-3ec4-45dc-9a5c-4ce74023751c admin admin] REQUEST_METHOD: `PUT` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:38.745789 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27301329-3ec4-45dc-9a5c-4ce74023751c admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:38.746064 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27301329-3ec4-45dc-9a5c-4ce74023751c admin admin] PATH_INFO: `/v3/projects/935988ddbe5941089a35137e84547ed5/users/66bd7e4131cc4c228f1ffb8ea6050863/roles/8fc75c351a0349e294deeed1ff391c4c` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:38.760604 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27301329-3ec4-45dc-9a5c-4ce74023751c admin admin] RBAC: Authorizing `identity:create_grant(project_id=935988ddbe5941089a35137e84547ed5, user_id=66bd7e4131cc4c228f1ffb8ea6050863, role_id=8fc75c351a0349e294deeed1ff391c4c)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:38.763280 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27301329-3ec4-45dc-9a5c-4ce74023751c admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:38.776952 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1431/2873] 10.222.0.45 () {60 vars in 1483 bytes} [Thu Aug 4 02:27:38 2022] PUT /identity/v3/projects/935988ddbe5941089a35137e84547ed5/users/66bd7e4131cc4c228f1ffb8ea6050863/roles/8fc75c351a0349e294deeed1ff391c4c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:27:38.787893 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf91a0-fcc4-4e0c-abe5-57960ba0cfe7 None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:38.788182 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf91a0-fcc4-4e0c-abe5-57960ba0cfe7 None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:38.788457 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf91a0-fcc4-4e0c-abe5-57960ba0cfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:38.817629 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-11cf91a0-fcc4-4e0c-abe5-57960ba0cfe7 None None] MFA Rules not processed for user `66bd7e4131cc4c228f1ffb8ea6050863`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:27:38.818851 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-11cf91a0-fcc4-4e0c-abe5-57960ba0cfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:27:38.881615 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1443/2874] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:27:38 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:27:38.893435 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27301329-3ec4-45dc-9a5c-4ce74023751c admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:38.895128 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-27301329-3ec4-45dc-9a5c-4ce74023751c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:27:38.915740 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:27:38.955502 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27301329-3ec4-45dc-9a5c-4ce74023751c admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:38.959122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0beaa3-4a6c-43c7-801f-56f785da83e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:38.959729 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0beaa3-4a6c-43c7-801f-56f785da83e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:38.960008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0beaa3-4a6c-43c7-801f-56f785da83e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:38.960269 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0beaa3-4a6c-43c7-801f-56f785da83e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:38.969875 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0beaa3-4a6c-43c7-801f-56f785da83e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:38.970928 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0beaa3-4a6c-43c7-801f-56f785da83e0 service cinder] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:38.979909 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1432/2875] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:27:38 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:27:46.032639 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf91a0-fcc4-4e0c-abe5-57960ba0cfe7 None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:46.099311 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf91a0-fcc4-4e0c-abe5-57960ba0cfe7 None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:46.103741 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ec5c0-aa33-4458-93b7-82cc5b9d652c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:46.104368 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ec5c0-aa33-4458-93b7-82cc5b9d652c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:46.104670 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ec5c0-aa33-4458-93b7-82cc5b9d652c admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:46.104949 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521ec5c0-aa33-4458-93b7-82cc5b9d652c admin admin] PATH_INFO: `/v3/users/66bd7e4131cc4c228f1ffb8ea6050863` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:46.106111 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ec5c0-aa33-4458-93b7-82cc5b9d652c admin admin] RBAC: Authorizing `identity:delete_user(user_id=66bd7e4131cc4c228f1ffb8ea6050863)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:46.108005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521ec5c0-aa33-4458-93b7-82cc5b9d652c admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:46.124463 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:27:46.155651 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-521ec5c0-aa33-4458-93b7-82cc5b9d652c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66bd7e4131cc4c228f1ffb8ea6050863'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:46.163017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-521ec5c0-aa33-4458-93b7-82cc5b9d652c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66bd7e4131cc4c228f1ffb8ea6050863'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:46.163670 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-521ec5c0-aa33-4458-93b7-82cc5b9d652c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66bd7e4131cc4c228f1ffb8ea6050863'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:46.175171 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-521ec5c0-aa33-4458-93b7-82cc5b9d652c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66bd7e4131cc4c228f1ffb8ea6050863'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:46.175793 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-521ec5c0-aa33-4458-93b7-82cc5b9d652c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66bd7e4131cc4c228f1ffb8ea6050863'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:46.188574 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1444/2876] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:27:46 2022] DELETE /identity/v3/users/66bd7e4131cc4c228f1ffb8ea6050863 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:27:46.193229 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6869a65-35c8-453e-8989-a57cb04489c8 None None] REQUEST_METHOD: `POST` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:46.193700 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6869a65-35c8-453e-8989-a57cb04489c8 None None] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:46.193816 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6869a65-35c8-453e-8989-a57cb04489c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:46.212841 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.auth.core [None req-b6869a65-35c8-453e-8989-a57cb04489c8 None None] MFA Rules not processed for user `1cdae040f69d4994bc484e82a80c2139`. Rule list: `[]` (Enabled: `True`). {{(pid=80673) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:27:46.214050 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-b6869a65-35c8-453e-8989-a57cb04489c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:27:46.278847 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1433/2877] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:27:46.289350 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ec5c0-aa33-4458-93b7-82cc5b9d652c admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:46.290801 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-521ec5c0-aa33-4458-93b7-82cc5b9d652c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:27:46.347439 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521ec5c0-aa33-4458-93b7-82cc5b9d652c admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:46.351087 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22accc-0ecf-426f-b2e1-a54cfa6f226c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:46.351688 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22accc-0ecf-426f-b2e1-a54cfa6f226c service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:46.351962 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22accc-0ecf-426f-b2e1-a54cfa6f226c service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:46.352242 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22accc-0ecf-426f-b2e1-a54cfa6f226c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:46.361025 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22accc-0ecf-426f-b2e1-a54cfa6f226c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:46.362429 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22accc-0ecf-426f-b2e1-a54cfa6f226c service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:46.372324 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1445/2878] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:27:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:27:46.640647 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6869a65-35c8-453e-8989-a57cb04489c8 None None] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:27:46.642761 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-b6869a65-35c8-453e-8989-a57cb04489c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:27:46.656045 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6869a65-35c8-453e-8989-a57cb04489c8 None None] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:27:46.662875 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d5ed0-40ab-42d9-8a31-a090b95dcc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:27:46.663527 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d5ed0-40ab-42d9-8a31-a090b95dcc03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:27:46.663805 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d5ed0-40ab-42d9-8a31-a090b95dcc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:27:46.664134 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d5ed0-40ab-42d9-8a31-a090b95dcc03 admin admin] PATH_INFO: `/v3/projects/935988ddbe5941089a35137e84547ed5` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:27:46.665262 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d5ed0-40ab-42d9-8a31-a090b95dcc03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=935988ddbe5941089a35137e84547ed5)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:27:46.667178 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d5ed0-40ab-42d9-8a31-a090b95dcc03 admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:27:46.715433 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-027d5ed0-40ab-42d9-8a31-a090b95dcc03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '935988ddbe5941089a35137e84547ed5', 'request_id': 'req-027d5ed0-40ab-42d9-8a31-a090b95dcc03'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:46.723371 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-027d5ed0-40ab-42d9-8a31-a090b95dcc03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '935988ddbe5941089a35137e84547ed5', 'request_id': 'req-027d5ed0-40ab-42d9-8a31-a090b95dcc03'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:46.729118 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-027d5ed0-40ab-42d9-8a31-a090b95dcc03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '935988ddbe5941089a35137e84547ed5', 'request_id': 'req-027d5ed0-40ab-42d9-8a31-a090b95dcc03'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:46.740005 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-027d5ed0-40ab-42d9-8a31-a090b95dcc03 admin admin] The token cache is being invalidate because project 935988ddbe5941089a35137e84547ed5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80673) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:27:46.740465 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-027d5ed0-40ab-42d9-8a31-a090b95dcc03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:46.740944 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-027d5ed0-40ab-42d9-8a31-a090b95dcc03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:27:46.742454 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1434/2879] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:27:46 2022] DELETE /identity/v3/projects/935988ddbe5941089a35137e84547ed5 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:27:56.367872 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:DELETE=9,keystone:SELECT=46,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:27:56.737008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:29:10.216191 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22accc-0ecf-426f-b2e1-a54cfa6f226c service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:29:10.217659 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-ba22accc-0ecf-426f-b2e1-a54cfa6f226c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:29:10.274565 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22accc-0ecf-426f-b2e1-a54cfa6f226c service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:29:10.322704 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25dc74d-0acd-4d3f-92f3-03a62e65fdca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f4b8838129174fbb8cdd32bd5d4d6101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:29:10.323319 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dc74d-0acd-4d3f-92f3-03a62e65fdca service cinder] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:29:10.323601 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dc74d-0acd-4d3f-92f3-03a62e65fdca service cinder] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:29:10.323882 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25dc74d-0acd-4d3f-92f3-03a62e65fdca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:29:10.325664 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-e25dc74d-0acd-4d3f-92f3-03a62e65fdca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:29:10.384268 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25dc74d-0acd-4d3f-92f3-03a62e65fdca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:29:10.385738 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25dc74d-0acd-4d3f-92f3-03a62e65fdca service cinder] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:29:10.395301 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1446/2880] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:29:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:29:14.564265 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d5ed0-40ab-42d9-8a31-a090b95dcc03 admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:29:14.565762 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-027d5ed0-40ab-42d9-8a31-a090b95dcc03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:29:14.628276 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d5ed0-40ab-42d9-8a31-a090b95dcc03 admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:29:14.631940 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178280c-2f47-40f6-8e9c-5c55989a1597 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:29:14.632549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178280c-2f47-40f6-8e9c-5c55989a1597 service neutron] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:29:14.632860 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178280c-2f47-40f6-8e9c-5c55989a1597 service neutron] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:29:14.633134 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178280c-2f47-40f6-8e9c-5c55989a1597 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:29:14.635409 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a178280c-2f47-40f6-8e9c-5c55989a1597 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:29:14.645149 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178280c-2f47-40f6-8e9c-5c55989a1597 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:29:14.646587 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178280c-2f47-40f6-8e9c-5c55989a1597 service neutron] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:29:14.655846 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1435/2881] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:29:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:29:18.330776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c2ddc-094f-4a36-8467-470d00bb7a8b None None] REQUEST_METHOD: `POST` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:29:18.331294 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c2ddc-094f-4a36-8467-470d00bb7a8b None None] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:29:18.331396 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c2ddc-094f-4a36-8467-470d00bb7a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:29:18.351374 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.auth.core [None req-928c2ddc-094f-4a36-8467-470d00bb7a8b None None] MFA Rules not processed for user `683f8f120df946f59505455a9ab2dc2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80672) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 04 02:29:18.353207 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-928c2ddc-094f-4a36-8467-470d00bb7a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:29:18.360021 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1447/2882] 10.222.0.45 () {58 vars in 958 bytes} [Thu Aug 4 02:29:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 04 02:29:19.117062 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178280c-2f47-40f6-8e9c-5c55989a1597 service neutron] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:29:19.118476 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-a178280c-2f47-40f6-8e9c-5c55989a1597 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:29:19.188293 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178280c-2f47-40f6-8e9c-5c55989a1597 service neutron] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:29:19.193299 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f49c9-ea24-499e-998f-3168d100eeea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796772cf491047c8b0a609dae9402307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:29:19.193915 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f49c9-ea24-499e-998f-3168d100eeea service placement] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:29:19.194197 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f49c9-ea24-499e-998f-3168d100eeea service placement] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:29:19.194471 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f49c9-ea24-499e-998f-3168d100eeea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:29:19.196906 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-cf8f49c9-ea24-499e-998f-3168d100eeea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:29:19.208091 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f49c9-ea24-499e-998f-3168d100eeea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:29:19.209689 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f49c9-ea24-499e-998f-3168d100eeea service placement] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:29:19.219708 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1436/2883] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:29:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:29:21.746311 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c2ddc-094f-4a36-8467-470d00bb7a8b None None] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:29:21.748312 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-928c2ddc-094f-4a36-8467-470d00bb7a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:29:21.822040 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=100 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:29:21.829085 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c2ddc-094f-4a36-8467-470d00bb7a8b None None] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:29:21.835158 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900eb6a-bb9d-4106-af4a-2ac23fd85a72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:29:21.835784 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900eb6a-bb9d-4106-af4a-2ac23fd85a72 service nova] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:29:21.836066 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900eb6a-bb9d-4106-af4a-2ac23fd85a72 service nova] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:29:21.836904 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900eb6a-bb9d-4106-af4a-2ac23fd85a72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:29:21.838714 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-7900eb6a-bb9d-4106-af4a-2ac23fd85a72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:29:21.857432 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900eb6a-bb9d-4106-af4a-2ac23fd85a72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:29:21.859047 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900eb6a-bb9d-4106-af4a-2ac23fd85a72 service nova] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:29:21.878866 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1448/2884] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:29:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:29:29.215348 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=58 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:29:31.868305 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=5 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:29:33.378900 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f49c9-ea24-499e-998f-3168d100eeea service placement] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:29:33.391617 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f49c9-ea24-499e-998f-3168d100eeea service placement] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:29:33.397093 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3764e-0d3e-4c7e-aad1-5a23cfa0eab6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:29:33.397704 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3764e-0d3e-4c7e-aad1-5a23cfa0eab6 service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:29:33.397983 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3764e-0d3e-4c7e-aad1-5a23cfa0eab6 service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:29:33.398721 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3764e-0d3e-4c7e-aad1-5a23cfa0eab6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:29:33.401632 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-fba3764e-0d3e-4c7e-aad1-5a23cfa0eab6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:29:33.414999 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3764e-0d3e-4c7e-aad1-5a23cfa0eab6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:29:33.416576 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3764e-0d3e-4c7e-aad1-5a23cfa0eab6 service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:29:33.427558 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1437/2885] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:29:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:29:43.422848 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=6 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:30:12.821278 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900eb6a-bb9d-4106-af4a-2ac23fd85a72 service nova] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:30:12.832395 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900eb6a-bb9d-4106-af4a-2ac23fd85a72 service nova] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:30:12.837979 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9147d9-84db-482a-816c-dfaa515ee8ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796772cf491047c8b0a609dae9402307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:30:12.838579 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9147d9-84db-482a-816c-dfaa515ee8ae service placement] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:30:12.838858 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9147d9-84db-482a-816c-dfaa515ee8ae service placement] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:30:12.839131 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9147d9-84db-482a-816c-dfaa515ee8ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:30:12.840881 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-5a9147d9-84db-482a-816c-dfaa515ee8ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:30:12.852811 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9147d9-84db-482a-816c-dfaa515ee8ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:30:12.854382 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9147d9-84db-482a-816c-dfaa515ee8ae service placement] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:30:12.864646 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1449/2886] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:30:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:30:22.860123 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=6 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:31:17.401878 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3764e-0d3e-4c7e-aad1-5a23cfa0eab6 service nova] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:31:17.403525 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-fba3764e-0d3e-4c7e-aad1-5a23cfa0eab6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:31:17.421870 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3764e-0d3e-4c7e-aad1-5a23cfa0eab6 service nova] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:31:17.428080 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee066c18-6341-4628-8136-5677f3c64cee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c571406d2a3f40dab035a918399d6f6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:31:17.428739 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee066c18-6341-4628-8136-5677f3c64cee service nova] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:31:17.429023 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee066c18-6341-4628-8136-5677f3c64cee service nova] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:31:17.429300 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee066c18-6341-4628-8136-5677f3c64cee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:31:17.432663 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-ee066c18-6341-4628-8136-5677f3c64cee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:31:17.446217 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee066c18-6341-4628-8136-5677f3c64cee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:31:17.447386 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee066c18-6341-4628-8136-5677f3c64cee service nova] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:31:17.459727 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1438/2887] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:31:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:31:27.454362 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=6 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:31:30.971302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9147d9-84db-482a-816c-dfaa515ee8ae service placement] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:31:30.981155 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9147d9-84db-482a-816c-dfaa515ee8ae service placement] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:31:30.984876 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e25af-b3be-4de0-8dec-8ad89b8598d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c2da427acb24c8cb6d667c6347858c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:31:30.985480 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e25af-b3be-4de0-8dec-8ad89b8598d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:31:30.985761 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e25af-b3be-4de0-8dec-8ad89b8598d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:31:30.986033 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3e25af-b3be-4de0-8dec-8ad89b8598d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:31:30.987808 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8c3e25af-b3be-4de0-8dec-8ad89b8598d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:31:31.049893 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e25af-b3be-4de0-8dec-8ad89b8598d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:31:31.051322 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3e25af-b3be-4de0-8dec-8ad89b8598d5 service neutron] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:31:31.061835 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1450/2888] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:31:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:31:34.217028 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee066c18-6341-4628-8136-5677f3c64cee service nova] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:31:34.229206 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee066c18-6341-4628-8136-5677f3c64cee service nova] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:31:34.234386 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecd50d-64d6-4c23-bd1f-2d9b9b5b024f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796772cf491047c8b0a609dae9402307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:31:34.234993 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecd50d-64d6-4c23-bd1f-2d9b9b5b024f service placement] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:31:34.235275 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecd50d-64d6-4c23-bd1f-2d9b9b5b024f service placement] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:31:34.235549 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ecd50d-64d6-4c23-bd1f-2d9b9b5b024f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:31:34.237311 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e8ecd50d-64d6-4c23-bd1f-2d9b9b5b024f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:31:34.250490 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecd50d-64d6-4c23-bd1f-2d9b9b5b024f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:31:34.252081 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ecd50d-64d6-4c23-bd1f-2d9b9b5b024f service placement] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:31:34.263305 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1439/2889] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:31:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:31:37.109595 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e25af-b3be-4de0-8dec-8ad89b8598d5 service neutron] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:31:37.124289 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3e25af-b3be-4de0-8dec-8ad89b8598d5 service neutron] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:31:37.130910 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba00f9-4219-4832-b1e3-eb1a911604e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796772cf491047c8b0a609dae9402307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:31:37.132093 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba00f9-4219-4832-b1e3-eb1a911604e9 service placement] REQUEST_METHOD: `GET` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:31:37.132972 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba00f9-4219-4832-b1e3-eb1a911604e9 service placement] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:31:37.133316 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba00f9-4219-4832-b1e3-eb1a911604e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:31:37.135617 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-3aba00f9-4219-4832-b1e3-eb1a911604e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:31:37.148533 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba00f9-4219-4832-b1e3-eb1a911604e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:31:37.151453 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba00f9-4219-4832-b1e3-eb1a911604e9 service placement] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:31:37.163333 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1451/2890] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:31:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:31:37.201745 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecd50d-64d6-4c23-bd1f-2d9b9b5b024f service placement] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:31:37.203321 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-e8ecd50d-64d6-4c23-bd1f-2d9b9b5b024f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:31:37.226018 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ecd50d-64d6-4c23-bd1f-2d9b9b5b024f service placement] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:31:37.230506 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45ba67-355a-4426-a8c9-ccd0a840fdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:31:37.231122 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45ba67-355a-4426-a8c9-ccd0a840fdab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:31:37.231402 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45ba67-355a-4426-a8c9-ccd0a840fdab admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:31:37.231676 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45ba67-355a-4426-a8c9-ccd0a840fdab admin admin] PATH_INFO: `/v3/users/683f8f120df946f59505455a9ab2dc2a` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:31:37.232853 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45ba67-355a-4426-a8c9-ccd0a840fdab admin admin] RBAC: Authorizing `identity:delete_user(user_id=683f8f120df946f59505455a9ab2dc2a)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:31:37.234755 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45ba67-355a-4426-a8c9-ccd0a840fdab admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:31:37.282618 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4e45ba67-355a-4426-a8c9-ccd0a840fdab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '683f8f120df946f59505455a9ab2dc2a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:37.283276 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4e45ba67-355a-4426-a8c9-ccd0a840fdab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '683f8f120df946f59505455a9ab2dc2a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:37.284231 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4e45ba67-355a-4426-a8c9-ccd0a840fdab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '683f8f120df946f59505455a9ab2dc2a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:37.297707 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4e45ba67-355a-4426-a8c9-ccd0a840fdab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '683f8f120df946f59505455a9ab2dc2a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:37.305428 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4e45ba67-355a-4426-a8c9-ccd0a840fdab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '683f8f120df946f59505455a9ab2dc2a'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:37.323008 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1440/2891] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:31:37 2022] DELETE /identity/v3/users/683f8f120df946f59505455a9ab2dc2a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:31:37.499168 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba00f9-4219-4832-b1e3-eb1a911604e9 service placement] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:31:37.510776 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba00f9-4219-4832-b1e3-eb1a911604e9 service placement] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:31:37.572922 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed07fc-cfea-4581-8df2-894be9904cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:31:37.573559 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed07fc-cfea-4581-8df2-894be9904cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:31:37.573835 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed07fc-cfea-4581-8df2-894be9904cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:31:37.574107 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed07fc-cfea-4581-8df2-894be9904cd0 admin admin] PATH_INFO: `/v3/projects/5f4683550a9d40d1ab0439d002d73749` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:31:37.575435 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed07fc-cfea-4581-8df2-894be9904cd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f4683550a9d40d1ab0439d002d73749)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:31:37.577358 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed07fc-cfea-4581-8df2-894be9904cd0 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:31:37.625642 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8bed07fc-cfea-4581-8df2-894be9904cd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f4683550a9d40d1ab0439d002d73749', 'request_id': 'req-8bed07fc-cfea-4581-8df2-894be9904cd0'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:37.633442 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8bed07fc-cfea-4581-8df2-894be9904cd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f4683550a9d40d1ab0439d002d73749', 'request_id': 'req-8bed07fc-cfea-4581-8df2-894be9904cd0'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:37.639983 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8bed07fc-cfea-4581-8df2-894be9904cd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f4683550a9d40d1ab0439d002d73749', 'request_id': 'req-8bed07fc-cfea-4581-8df2-894be9904cd0'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:37.661744 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8bed07fc-cfea-4581-8df2-894be9904cd0 admin admin] The token cache is being invalidate because project 5f4683550a9d40d1ab0439d002d73749 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:31:37.662042 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8bed07fc-cfea-4581-8df2-894be9904cd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:37.662731 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8bed07fc-cfea-4581-8df2-894be9904cd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:37.664787 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1452/2892] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:31:37 2022] DELETE /identity/v3/projects/5f4683550a9d40d1ab0439d002d73749 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 04 02:31:37.668918 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45ba67-355a-4426-a8c9-ccd0a840fdab admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:31:37.670515 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-4e45ba67-355a-4426-a8c9-ccd0a840fdab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:31:37.684354 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45ba67-355a-4426-a8c9-ccd0a840fdab admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:31:37.740062 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c559d9f-40cd-4067-a487-544525b1599b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:31:37.740659 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c559d9f-40cd-4067-a487-544525b1599b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:31:37.740930 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c559d9f-40cd-4067-a487-544525b1599b admin admin] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:31:37.741204 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c559d9f-40cd-4067-a487-544525b1599b admin admin] PATH_INFO: `/v3/users/9637ce56ef2b4e41a3d485ff565a1ee4` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:31:37.743023 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c559d9f-40cd-4067-a487-544525b1599b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9637ce56ef2b4e41a3d485ff565a1ee4)` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:31:37.744964 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c559d9f-40cd-4067-a487-544525b1599b admin admin] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:31:37.798899 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4c559d9f-40cd-4067-a487-544525b1599b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9637ce56ef2b4e41a3d485ff565a1ee4'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:37.799800 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4c559d9f-40cd-4067-a487-544525b1599b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9637ce56ef2b4e41a3d485ff565a1ee4'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:37.800393 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4c559d9f-40cd-4067-a487-544525b1599b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9637ce56ef2b4e41a3d485ff565a1ee4'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:37.810805 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:31:37.813160 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4c559d9f-40cd-4067-a487-544525b1599b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9637ce56ef2b4e41a3d485ff565a1ee4'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:37.822123 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.notifications [None req-4c559d9f-40cd-4067-a487-544525b1599b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9637ce56ef2b4e41a3d485ff565a1ee4'} {{(pid=80673) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:37.834989 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1441/2893] 10.222.0.45 () {58 vars in 1224 bytes} [Thu Aug 4 02:31:37 2022] DELETE /identity/v3/users/9637ce56ef2b4e41a3d485ff565a1ee4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:31:38.011329 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed07fc-cfea-4581-8df2-894be9904cd0 admin admin] Authenticating user token {{(pid=80672) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:31:38.013005 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.fernet_utils [None req-8bed07fc-cfea-4581-8df2-894be9904cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80672) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:31:38.030303 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed07fc-cfea-4581-8df2-894be9904cd0 admin admin] Validating token access rules against request {{(pid=80672) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:31:38.040495 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:31:38.097498 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e7901-f451-4475-908f-09c4810e4461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cdae040f69d4994bc484e82a80c2139', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '457cd62a685649778d5130f3937b1bcf', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80672) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:31:38.098138 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e7901-f451-4475-908f-09c4810e4461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:31:38.098420 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e7901-f451-4475-908f-09c4810e4461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:31:38.098695 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e7901-f451-4475-908f-09c4810e4461 admin admin] PATH_INFO: `/v3/projects/532331a91f4f4c7283dbbcbce8804160` {{(pid=80672) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:31:38.099848 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e7901-f451-4475-908f-09c4810e4461 admin admin] RBAC: Authorizing `identity:delete_project(project_id=532331a91f4f4c7283dbbcbce8804160)` {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:31:38.102410 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e7901-f451-4475-908f-09c4810e4461 admin admin] RBAC: Authorization granted {{(pid=80672) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:31:38.148249 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8f4e7901-f451-4475-908f-09c4810e4461 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '532331a91f4f4c7283dbbcbce8804160', 'request_id': 'req-8f4e7901-f451-4475-908f-09c4810e4461'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:38.155610 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8f4e7901-f451-4475-908f-09c4810e4461 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '532331a91f4f4c7283dbbcbce8804160', 'request_id': 'req-8f4e7901-f451-4475-908f-09c4810e4461'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:38.161477 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8f4e7901-f451-4475-908f-09c4810e4461 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '532331a91f4f4c7283dbbcbce8804160', 'request_id': 'req-8f4e7901-f451-4475-908f-09c4810e4461'} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:38.173017 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8f4e7901-f451-4475-908f-09c4810e4461 admin admin] The token cache is being invalidate because project 532331a91f4f4c7283dbbcbce8804160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80672) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 04 02:31:38.173302 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8f4e7901-f451-4475-908f-09c4810e4461 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:38.173934 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG keystone.notifications [None req-8f4e7901-f451-4475-908f-09c4810e4461 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80672) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 04 02:31:38.175444 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: [pid: 80672|app: 0|req: 1453/2894] 10.222.0.45 () {58 vars in 1233 bytes} [Thu Aug 4 02:31:38 2022] DELETE /identity/v3/projects/532331a91f4f4c7283dbbcbce8804160 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 04 02:31:47.831618 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:DELETE=2,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:31:48.170002 dv-cinder-740146-17-iscsi devstack@keystone.service[80672]: DEBUG dbcounter [-] [80672] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80672) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 04 02:35:18.791191 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c559d9f-40cd-4067-a487-544525b1599b admin admin] Authenticating user token {{(pid=80673) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 04 02:35:18.792605 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-4c559d9f-40cd-4067-a487-544525b1599b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:35:18.930975 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c559d9f-40cd-4067-a487-544525b1599b admin admin] Validating token access rules against request {{(pid=80673) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 04 02:35:18.988992 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c41d44-87d6-4ebb-9c76-ee99ff42c507 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '796772cf491047c8b0a609dae9402307', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '74b8caff828a4f5289094fb2ae2278f7', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80673) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 04 02:35:18.989607 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c41d44-87d6-4ebb-9c76-ee99ff42c507 service placement] REQUEST_METHOD: `GET` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 04 02:35:18.989871 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c41d44-87d6-4ebb-9c76-ee99ff42c507 service placement] SCRIPT_NAME: `/identity` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 04 02:35:18.990172 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c41d44-87d6-4ebb-9c76-ee99ff42c507 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80673) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 04 02:35:18.991909 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.fernet_utils [None req-44c41d44-87d6-4ebb-9c76-ee99ff42c507 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80673) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 04 02:35:19.003877 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c41d44-87d6-4ebb-9c76-ee99ff42c507 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 04 02:35:19.005497 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c41d44-87d6-4ebb-9c76-ee99ff42c507 service placement] RBAC: Authorization granted {{(pid=80673) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 04 02:35:19.016102 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: [pid: 80673|app: 0|req: 1442/2895] 10.222.0.45 () {60 vars in 1365 bytes} [Thu Aug 4 02:35:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 04 02:35:29.011263 dv-cinder-740146-17-iscsi devstack@keystone.service[80673]: DEBUG dbcounter [-] [80673] Writing DB stats keystone:SELECT=89 {{(pid=80673) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}